UsersQDir setting

UsersQDir setting SearchSearch
Author Message
ashot shahbazian
New member
Username: Animatele

Post Number: 24
Registered: 06-2004
Posted on Friday, July 10, 2009 - 03:10 am:   

Hi Bryce, Des,

Tried the UsersQDir=d:\path\ setting today, it seems to be not working. The user queues stayed where they were, beneath the \NowSMS program directory. Nothing appeared in the new folder (I assumed there should have been "username" sub-folders there.) I tried UsersQDir=d:\path\ with and without a backslash in the end and restarted the service each time I made the change. FYI, similar LogDirectory= and QDir= settings only work if there's no backslash in the end.

Also, every time I'd make a change to the UsersQDir= setting in the smsgw.ini file and save it the server's CPU would almost lock without much traffic going through. When I finally removed it and stopped/started the service the CPU load dropped to normal.

I've not tried applying the UsersDir= setting, just the UsersQDir= alone.

Can you please check if you are able to use the UsersDir and UsersQDir with the latest release/patches? Do you think the reason could be that we've only created the path on the new volume, but not the user-specific subdirectories? If they are needed beforehand, what should be the directory structure?

Questions about the UsersDir setting:

- if we tried the UsersDir setting, where would the files keeping user account stats move to? Do we have to move the existing files manually to the new user subdirectories before applying the change and starting the service - so that the stats are not lost of corrupted? If we do, then we need to create the directory structure beforehand. Would the structure be \newusersdir\usernameX identical to \users\usernameX beneath \NowSMS without the setting?

- What is the purpose of the SMSUSERS.D2A and SMSUSERS.D2I files? Should we also move them from \NowSMS\ to the new location before applying the change to ensure continuity? If so, where to exactly? Any other files, apart from current day logs, need to be moved to the new location before applying the changes?

Kind regards,
Ashot
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1023
Registered: 08-2008
Posted on Friday, July 10, 2009 - 07:06 pm:   

Hi Ashot,

I almost replied to you earlier, but this required more investigation.

It appears that there is no "UsersQDir=" setting.

It was added in an interim release in 2008.11. But it was removed shortly thereafter because one of our engineers thought it was a typo, and that it should have been "UsersDir=" instead.

"UsersQDir=" definitely has no effect in the 2009.07 releases, while "UsersDir=" seems to be working fine.

I tested by adding the setting and only moving over the SMSUSERS.D2A/D2I files. User specific subdirectories seemed to be created as needed.

The SMSUSERS.D2A/D2I files contain the user account information that is checked when an account connects.


quote:

if we tried the UsersDir setting, where would the files keeping user account stats move to?




newusersdir\username


quote:

Would the structure be \newusersdir\usernameX identical to \users\usernameX beneath \NowSMS without the setting?




Yes, the structure is identical.

You could use XCOPY /s against the current users directory to move all of the files.


quote:

What is the purpose of the SMSUSERS.D2A and SMSUSERS.D2I files? Should we also move them from \NowSMS\ to the new location before applying the change to ensure continuity? If so, where to exactly?




Yes. Those files go into newusersdir directly.


quote:

Any other files, apart from current day logs, need to be moved to the new location before applying the changes?




It wouldn't hurt to do an XCOPY to move all the files.

--
Des
NowSMS Support
ashot shahbazian
New member
Username: Animatele

Post Number: 25
Registered: 06-2004
Posted on Friday, July 10, 2009 - 08:44 pm:   

Hi Des,

Thanks for a comprehensive reply!

Are there plans to add the "UsersQDir=" setting again? Both settings were in fact requested by us in our earlier communications, in odrer to free the program disk from being frequently accessed. In our system, we keep the logs and the queues (there is a lot more of them created by other components) on two shared extremely fast NAS volumes, one for logs and the other for queues. This prevents disk I/O bottlenecks that slow down the servers. It'd be great if NowSMS would allow this too, as we don't like keeping logs and queues on the same volume. The one that contains the program directory now shows a much larger average disk queue size than all other volumes combined, possibly because of both NowSMS user logs and queues handled by a relatively slow local mirror set.

Kind regards,
Ashot
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1031
Registered: 08-2008
Posted on Monday, July 13, 2009 - 07:55 pm:   

Hi Ashot,

I'm going to have to follow-up with you in a few days on this.

Apparently there was a problem with the implementation of the setting, and the fix was to assume that it was a typo and remove it.

I think it can be re-implemented, but we need to do some testing because no one remembers what the problem was.

I'll make a note to follow-up in a few days.

--
Des
NowSMS Support
ashot shahbazian
New member
Username: Animatele

Post Number: 30
Registered: 06-2004
Posted on Tuesday, July 14, 2009 - 05:32 am:   

Hi Des,

Excellent! One more similar useful feature would also be the ability to set a separate path for an "SMS-IN" folder, as that's in fact where every inbound message is processed first.

We've set the UserDir to a separate fast volume, it did help a lot. The write IOPS and the disk queue on a slow program disk had decreased about 3-fold, which made the system noticeably more responsive.

Kind regards,
Ashot.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1042
Registered: 08-2008
Posted on Tuesday, July 14, 2009 - 03:44 pm:   

Hi Ashot,

SmsInDir= should be able to set a location for this directory.

Note, however, that not every inbound message is processed there first.

That was largely true at one point in time.

However, one of the performance enhancements that we made over the past year was to better optimise message flow.

By speeding up the way we evaluate how to process the received message, it no longer has to be queued in SMS-IN first. If there's a receipt match, it goes directly into the user queue. If there's a "ReRouteReceived" it goes directly to the outbound queue.

We probably should have been doing that all along. However, previously the lookups to determine the routing to apply slowed down the message processing on the SMPP connection, so we queued for a separate lookup (which caused other bottlenecks).

--
Des
NowSMS Support
Nikos Mavrakis
New member
Username: Nmavra

Post Number: 11
Registered: 05-2007
Posted on Wednesday, July 15, 2009 - 02:32 pm:   

Hi Des, you said
"By speeding up the way we evaluate how to process the received message, it no longer has to be queued in SMS-IN first. If there's a receipt match, it goes directly into the user queue. If there's a "ReRouteReceived" it goes directly to the outbound queue. "

By "receipt match" do you perhaps mean delivery reports? Because I still see .rec files in my SMS-IN directory, you may also remember us having issues with delivery reports a few days ago.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1046
Registered: 08-2008
Posted on Wednesday, July 15, 2009 - 05:54 pm:   

Hi Nikos,

I believe you're using NowSMS a little differently.

The scenario that I'm referring to here is when an SMPP client submits a message to NowSMS requesting a delivery receipt.

NowSMS routes the message upstream (usually to another SMPP connection, but it could be a CIMD or UCP connection).

When the delivery receipt comes back ... in this particular scenario ... NowSMS needs to route the message back to the original SMPP client.

In previous versions of NowSMS, NowSMS used to first store the received message in the SMS-IN directory ... and then the NowSMS 2-way processing would re-queue the message for the original SMPP client.

That added a considerable amount of overhead for that particular usage scenario.

I think the way you use NowSMS is different.

Messages that are destined for 2-way command processing are still queued in the SMS-IN directory.

*.IN files are regular message files.

*.REC files are new in NowSMS 2009, and they are receipts.

*.RCT files are unresolved receipts (receipts where we cannot correlate the Message ID).

NowSMS 2009 processes these three queues independently.

The idea is that a large number of receipts should not slow down regular inbound message processing.

Similarly, NowSMS throttles processing of regular inbound messages (*.IN) based upon the message license speed. By processing receipts independently, receipt processing does not get blocked by license limit throttling, like it could in earlier versions.

The crazy thing is that there are so many different usage scenarios. But both of these scenarios have seen major performance improvements in recent releases. Ashot's scenario has seen a much more dramatic performance improvement because we have greatly reduced both disk i/o and processor overhead.

In your case, *.REC files should not be present for very long unless there are delays in how quickly your 2-way commands process messages.

--
Des
NowSMS Support
ashot shahbazian
New member
Username: Animatele

Post Number: 32
Registered: 06-2004
Posted on Wednesday, July 15, 2009 - 06:02 pm:   

Hi Nikos,

As far as we can tell, the .rec files appear there for the DLR that were not matched on-the fly. if you watch one, you'll notice most renaming into .sms files which then stay in \sms-in forever.

I suggest you check for the ReceiptMessageID field in these files (in a separate line) and check the actual ID values in the file against those in corresponding outbound messages in the smsout log. If you find that there's no ReceiptMessageID, or the ID-s don't match to those in logs, or that ones are in hex and others in decimal, that'd be the likely reason why they don't route where intended.

Kind regards,
Ashot
Nikos Mavrakis
New member
Username: Nmavra

Post Number: 17
Registered: 05-2007
Posted on Thursday, July 16, 2009 - 10:24 am:   

Des the .rec files I still see in our SMS-IN directory are usually handled fast, depending on how one defines "fast".

In our example, with normal sms traffic at any given time we see 10-15 .rec files in the folder. Every time we refresh, we still see 10-15 but they are usually 10 same ones and 5-6 new. Some stay there for a few seconds, and some others leave almost immediately.


My question is what is our 2-way commands' speed dependant on, and how can we improve this? Is there something we can do on our web server, perhaps IIS tuning, windows 2003 tuning etc? Is it hardware dependant?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1064
Registered: 08-2008
Posted on Thursday, July 23, 2009 - 03:21 pm:   


quote:

My question is what is our 2-way commands' speed dependant on, and how can we improve this? Is there something we can do on our web server, perhaps IIS tuning, windows 2003 tuning etc? Is it hardware dependant?




Hi Nikos,

I apologise for not following up with you on this.

It's hard to say for certain.

I've never seen a situation where we needed to do any performance tweaking or tuning on the web server itself. If there is an issue, it's usually in the script itself ... the speed at which the script itself processes the request is generally much more significant than any other factor.

Can you enable the SMSDEBUG.LOG for 30 minutes or so?

Then, I'd like to get a copy of that log so that we can analyse it, and get a better feel of the timings. If you can put it into a ZIP or RAR file under 10MB ... e-mail it to nowsms@nowsms.com with "Attention: Des" in the subject line.

Also, what is your license speed? The ".IN" files will be throttled based upon the license speed, but the ".REC" files are not throttled. They should be processed as quickly as your script can process them.

--
Des
NowSMS Support
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1172
Registered: 08-2008
Posted on Wednesday, August 26, 2009 - 05:38 pm:   

Follow-up regarding the UsersQDir setting. This setting was restored for the 2009.08.26 update at http://www.nowsms.com/download/nowsmsupdate.zip.

--
Des
NowSMS Support