NowSMS Update available for Test Release

NowSMS Update available for Test Release SearchSearch
Author Message
Keith Norris
Moderator
Username: Admin

Post Number: 36
Registered: 05-2003
Posted on Wednesday, November 05, 2008 - 06:44 pm:   

Over the past several months, we have been working on some optimisations and performance enhancements to the core message routing logic of the NowSMS product.

These enhancements have been geared primarily toward supporting higher volume messaging environments with speed requirements of 400 messages per second and higher.

These core performance enhancements are also of interest to other high speed, but not-quite-as-high-speed environments for a variety of reasons.

1.) Requesting delivery receipts for all (or most) messages can put an increased strain on the system, with previous versions of NowSMS peaking at speeds of 150 to 200 messages per second, and sometimes with a queued backlog of receipt processing. Performance is easily doubled in this update, and the queue backlog is no more.

2.) 2-way SMS command performance has been optimised, particularly for HTTP based commands, allowing an HTTP based command to easily keep up with burst rates of hundreds of messages per second.

3.) CPU load with large numbers of outbound messaging routes. Historically, the more outbound routes defined to NowSMS, the higher the CPU load, especially if there is a large volume of queued messages waiting for a small number of routes. The routing logic has been dramatically improved to reduce CPU load.

4.) If you use HTTP-based accounting callbacks for billing or accounting, they may be significantly limiting your overall message throughput. The new release uses HTTP keep-alive sockets and optimised logic to improve throughput when accounting callbacks are enabled.

There are also a number of new features in this release. A list of features and enhancements from the readme file can be found at the end of this message. However, the primary focus of this NowSMS update is performance.

We've completed our internal testing of this update. However, because there were a large number of internal changes, optimisations and fixes, we are going to make this version available as a test release before general release.

We're quite confident that this release is actually more stable and robust than previous versions. We know the performance and CPU overhead is much improved. We know that overall disk I/O has been reduced. And we know several rather significant bugs that have been fixed in this release. But, the problem with bugs is they are very much configuration dependent, and with such a large number of configuration options, we may have missed something that affects articular configurations, so we want to get some more feedback.

For this reason, we're going to go ahead and make this update available in a testing release prior to a more general release.

This update can be downloaded at http://www.nowsms.com/download/nowsms200811.zip.

More information from the readme file:

2008-11-05:

* MMSC: Add configuration setting for "MMSC VASP" to allow "Force via Defined Route" to be set to "Direct Delivery". When "Direct Delivery" is selected for this setting, this means that any MMS messages received from this route will be queued for direct delivery with NowSMS as the MMSC, ignoring any other routing considerations.

* SMS Gateway: Add ability for SMS accounting callbacks to be used to specify routing information. When the "SMSSend" accounting callback is issued, it is possible for the callback to return SMSCRoute=routename in the HTTP response to specify the route via which the message should be routed. http://support.nowsms.com/discus/messages/1/23919.html.

* SMS Gateway: To improve performance, keep-alive sockets are now enabled by default for SMS accounting callbacks. To disable this support, edit SMSGW.INI, and under the [SMSGW] header, add AccountingKeepAlive=No.

* SMS Gateway: To improve performance, keep-alive sockets are now enabled by default for 2-way SMS callbacks. To disable this support, edit SMSGW.INI, and under the [SMSGW] header, add 2WayKeepAlive=No. See http://support.nowsms.com/discus/messages/1/24486.html

* SMS Gateway: For licenses of 10 messages per second or higher, default to a higher setting for 2WaySMSThreadCount=##, so that more 2-way command processing threads are allocated by default, allowing multiple 2-way commands to be processed simultaneously.

* E-Mail to SMS: Allow e-mail format of sms.phonenumber@domain.com for a customer requirement.

* SMS Gateway: For for server restart after midnight due to delivery receipt tracking bug introduced in 2008.08.22.

* MMSC: Fix for high CPU utilisation problem that could occur when upgrading from older MMSC versions, especially if MM4 interconnects are defined.

* SMS Gateway: Add support for specifying that SMS messages with particular SMPP "service_type" values should be routed via specific SMSC connections. In the "Preferred SMSC Connection for" list, it is possible to specify service_type values, in addition to recipeint address masks. To specify that messages with the service_type value of "bulk" should be routed via a connection, add "service_type=bulk" to the "Preferred SMSC Connection for" list. To specify that messages with a service_type of "bulk" should be blocked from routing via the connection, add "!service_type=bulk" the "Preferred SMSC Connection for" list. If one or more "service_type=" entries is defined on a "Preferred SMSC Connection for" list, then only messages that match a listed service type value will be routed via this connection. Recipient masks can also be included in the list, however when a service type value is also defined, only messages that match a listed service type value and a recipient address mask, will be routed via this connection. (Note: It is also possible to specify a "service_type" value when submitting messages via HTTP by including "&ServiceType=xxxxxx" in the URL parameter.) Similarly, it is possible to include "service_type=xxxx" in the "Recipient address(es) to route to this user" attribute of an "SMS Users" account to force messages addressed to a particular service_type to be queued for delivery to an SMPP client. See http://support.nowsms.com/discus/messages/1/24460.html.

* SMS Gateway/SMPP Client: Update configuration parameter that allows the "users" directory to be moved to another location other than beneath the NowSMS program directory. This directory contains queues of received messages waiting for SMPP or POP3 clients, as well as user specific log files. Under the [SMSGW] header in SMSGW.INI, use UsersDir=d:\path\ or UsersDir=\Server\Volume\path\ to specify the location of the SMS "users" directory. When this parameter is set, the "SMS Users" database files (SMSUSERS.D2A and SMSUSERS.D2I) will also be located in this directory (by default, when this parameter is not set, these files are in the NowSMS program directory). With this release, it is now possible to separate the queued messages from other user related data and logs. Under the [SMSGW] header, use UsersQDir=d:\path\ or UsersQDir=\Server\Volume\path\ to specify the location of the user directories that will hold queued messages only. Other user directories and files will be stored relative to the "UsersDir=" setting, or in their default location if "UsersDir=" is not present.

* MMSC Routing: Fix for a strange problem on Windows Vista and Windows Server 2008, where under certain conditions the user interface would make changes to the configuration of an "MMSC Routing" definition, but these changes would not be seen by the back-end MMSC service. This could happen because of an administrative rights issue. When the NowSMS configuration program is launched after installation, it is usually running with administrative rights, because administrative rights are required by the installation program, and these rights are inherited by the configuration program when it is launched at the end of the installation. Other times, the NowSMS configuration program is launched without requiring administrative rights, except for installing or starting/stopping services. If an "MMSC Routing" definition is initially defined by the configuration program running with administrative rights, settings for this "MMSC Routing" definition can only be modified if the configuration program is running with administative rights. This fix resolves the problem for new "MMSC Routing" definitions, however it does not resolve the problem for existing installations. If a problem is suspected in an existing installation, try to launch the NowSMS configuration program with a "right-click" and "Run as Administrator" to see if different settings are shown for the "MMSC Routing". If different settings are found, the only resolution for an existing installation is to delete ProgramData\NowSMS\VASPOUT\VASP.INI and recreate the "MMSC Routing", preferably with the configuration program not running with administrative rights.

* SMS Gateway: Performance optimisations for routing logic that evaluates "Preferred SMSC Connection for" recipient address masks.

* SMS Gateway: Add support for specifying blocked recipient address masks for individual SMSC connections. In the "Preferred SMSC Connection for" list, it is possible to specify recipeint address masks that should not be routed via this connection, in addition to recipient address masks that should be routed via this connection. To specify a recipient address mask that should be blocked from using this connection, prefix the address mask with "!". For example, if +447* and !+447700* are in the "Preferred SMSC Connection for" list, +447123456789 would be routed via this connection, but +447700123456 would not be routed via this connection.

* Work-around for problem occurring on some systems where the NowSMS trial period only lasts for a single day. A popular virus scanner product is blocking part of the install program, so that the trial period counter is not properly initialised.

* SMS Gateway: Delete any stray *.LCK files that remain in the Q directory (shouldn't happen unless Q files are manipulated by a separate program external to NowSMS). Make the time period that *.ERR files remain in the Q directory configurable, with the default being 3 days. To change this time period, edit SMSGW.INI, and under the [SMSGW] header, add ErrorQRetainDays=##.

* SMS Gateway: Update SQLITE.DLL used for maintaining SMPP delivery receipt tracking databases, and limit the amount of memory used to cache the databases so that NowSMS does not take up an excessive amount of memory on extremely busy servers.

* SMS Gateway: Faster routing of delivery receipts back to the submitting user account as an interim internal routing step is now bypassed.

* SMS Gateway: Add configuration option to allow selected "SMPPOptions" settings to be automatically copied when generating a reply via the 2-way SMS command facility. Under the [SMSGW] header of SMSGW.INI, the 2WayReplyCopySMPPOptions=xxxx,yyyy setting can be used to specify a list of "SMPPOptions" TLV paramters that should be copied from the received message to the reply message generated by the 2-way command. Any options specified in this list must be included in the [SMPPOptions] section of SMSGW.INI. See http://support.nowsms.com/discus/messages/1/24487.html

* SMS Gateway/CIMD: Fix for 6 to 7 second delays in-between message submissions when messages are received over the same connection.

* SMS Gateway/SMPP Server: Fix for a bug where when NowSMS is acting as an SMPP server, if NowSMS tries to deliver an SMS message to a connected client, but the client does not acknowledge the message, and the client continues to send enquire_link packets, NowSMS will not attempt to deliver any other messages to the client. See http://support.nowsms.com/discus/messages/1/24483.html.

* SMS Gateway: Experimental - Add configuration option to allow the "SMSCRoute=" paramter to be used by SMPP client connections (e.g.,"SMS Users" connected to NowSMS via SMPP). Under an [SMPP] section header of SMSGw.INI, it is possible to add SMSCRouteTLV=####, where #### is the tag number (in hexadecimal). When NowSMS routes a message to an SMPP client, this TLV parameter will contain the route information for the SMSC route from which the message was received (either the SMSC Host Name and port, or the RouteName= parameter configured in SMSGw.INI for the SMSC). Similarly, the SMPP client can specify this parameter when submitting a message to request a specific outbound route for routing the message. See http://support.nowsms.com/discus/messages/1/24485.html.

2008-10-23:

* SMS Gateway/GSM Modem: Add additional modem test for proper implementation of the command that is used for sending SMS messages. Some modems return OK responses to commands that they do not understand, NowSMS now performs further checks to make sure that the modem actually understands all required commands. Previously NowSMS would return OK responses for some Blackberry devices, even though the devices did not implement the command correctly. Also, some of the virtual COM ports defined by the Sierra Wireless 3G devices exhibit this same behaviour. This check helps ensure that NowSMS is not configured to use a modem port that does not provide the required support. The new error message is "Modem does not properly support command for sending SMS (AT+CMGS)".

* SMS Gateway/GSM Modem: When using NowSMS with some modems, such as Sierra Wireless devices with the "Watcher" software enabled, other software that ships with the modem will process received messages before the messages are presented to NowSMS. NowSMS now polls the modem for both read and unread messages so that all messages will be processed. To disable this behaviour for a particular modem, edit SMSGW.INI, and under the [Modem - Driver Name] section header, add ProcessReadMessages=No.

* SMS Gateway/GSM Modem: The "SMS Message Storage" = "Default" receive SMS message handler has been updated to support a mix of polling and direct delivery. This is done primarily to resolve problems with new installations where NowSMS could not receive any messages from the modem until "SMS Message Storage" was set to the "Direct to Modem" setting. If upgrading from a previous version, and a problem is experienced with receiving SMS messages, change "SMS Message Storage" to the "Device Memory" setting for the "Default" setting to work as in previous versions.



2008-10-22:

* MMSC: Add support for outbound message throttling limits for sending messages via external "MMSC Routing" connections (e.g., MM7, MM4, MM1, EAIF). These limits can be defined by manually editing the VASPOUT\accountname\VASP.INI file, and under the [VASP] header, adding SendLimit=x/y where "x" is the number of messages that can be sent over a "y" second interval. If "y" is omitted, a value of "1" is assumed. For example, SendLimit=3 limits the connection to a speed of 3 messages per second. SendLimit=1/2 limits the connection to a speed of 1 message every 2 seconds.


2008-10-17:

* SMS Gateway/SMPP: Fix for potential "exception error" introduced in 2008 version, that could, under limited circumstances, when the debug log is active, trigger a NowSMS service restart when an SMPP connection teriminates unexpectedly.

2008-10-16:

* SMS Gateway/SMPP Server: Fix for a problem where if multiple receiver (or transceiver) binds existed for the same "SMS Users" account, duplicate messages could be delivered to the client.

* MMSC/MM4: Increase maximum supported length for X-Mms-Transaction-ID header from 99 to 249 characters. A customer connecting to France Telecom Open Transit MMS Interconnect was encountering extremely long transaction ids.

* SMS Gateway: Fix for situation where NowSMS would establish only a "receiver bind", even though "Receive SMS Messages" was not checked for the connection. This would happen if 1.) "Support any outbound message traffic" is NOT checked. 2.) The "Preferred SMSC Connection for" list is empty.
3.) Under "Advanced Settings", "Send and Receive Messages over the same connection" is NOT checked. It should happen only if these three conditions are met, and the "Receive SMS Messages" setting is also checked. For more information see http://blog.nowsms.com/2008/07/smpp-connection-types-sender-receiver.html and http://support.nowsms.com/discus/messages/1/24451.html.

2008-10-13:

* MMSC: Internal changes to migrate the VASPQ directory structure to use multiple subdirectories to better facilitate larger message queues and higher volumes of MMS message routing to external MMSC connections. This capability is only enabled by default on 5 messages per second or higher licenses. To enable or disable, edit MMSC.INI, and under the [MMSC] section header, add EnableBulkVASPQ=Yes (or No).

2008-10-09:

* 2-way SMS: Enhance mailto: command support for e-mail to SMS in the 2-way command facility by supporting additional parameters including cc, subject and body. For example, mailto:nowsms@nowsms.com?cc=nowsms@nowsms.net&subject=Received%20SMS%20Message&body=@@FULLSMS@@. See http://support.nowsms.com/discus/messages/1/24428.html.

2008-10-07:

* PAP/PPG interface: Add support for "?SMSCRoute=routname" parameter in PAP URL to specify explicit SMSC routing information. See http://support.nowsms.com/discus/messages/1/24429.html.

2008-10-01:

* MMSC/MM7: Remove trailing "-" characters from MIME boundary separator in MM7 postings, as this appears to be causing a problem with a particular mobile operator MMSC.

* UCP/EMI: Add configuration option to specify UCP/EMI error codes that should be considered a permanent error, so that NowSMS will not retry submitting the message. To specify error codes that should be considered as a permanent error, edit SMSGW.INI, and under the [SMSGW] header, add "UCPRejectErrorCodes=x,y,z". This value can contain a comma delimited list of UCP/EMI error codes. Values should be specified in decimal, e.g., "UCPRejectErrorCodes=21,23,24,25". If you are unsure what error code value your provider is returning for particular error situations, check the error codes listed in the NowSMS SMSOUT-yyyymmdd.LOG file. Alternatively, it is also possible to specify "UCPRejectErrorCodes=All", which indicates that all error codes should be considered a permanent error, so that NowSMS will not retry submitting the message.

* CIMD: Add configuration option to specify CIMD error codes that should be considered a permanent error, so that NowSMS will not retry submitting the message. To specify error codes that should be considered as a permanent error, edit SMSGW.INI, and under the [SMSGW] header, add "CIMDRejectErrorCodes=x,y,z". This value can contain a comma delimited list of CIMD error codes. Values should be specified in decimal, e.g., "CIMDRejectErrorCodes=300,301,302". If you are unsure what error code value your provider is returning for particular error situations, check the error codes listed in the NowSMS SMSOUT-yyyymmdd.LOG file. Alternatively, it is also possible to specify "CIMDRejectErrorCodes=All", which indicates that all error codes should be considered a permanent error, so that NowSMS will not retry submitting the message.

* SMPP: (This configuration option has existed since prior to NowSMS 2006, except for the SMPPRejectErrorCodes=All option which was added in this release.) Add configuration option to specify SMPP error codes that should be considered a permanent error (NowSMS will not retry the message). By default, NowSMS treats the following SMPP error codes as a permanent error: A (ESME_RINVSRCADR - invalid source address), B (ESME_RINVDSTADR - invalid destination address), 66 (ESME_RX_R_APPN - ESME Receiver Reject Message). To add additional error codes that should be considered as a permanent error, edit SMSGW.INI, and under the [SMSGW] header, add "SMPPRejectErrorCodes=x,y,z". This value can contain a comma delimited list of SMPP error codes. Values should be specified in hex, with no leading zeroes, e.g., "SMPPRejectErrorCodes=A,B,66". Alternatively, it is also possible to specify "SMPPRejectErrorCodes=All", which indicates that all error codes, except ESME_RTHROTTLED and ESME_RMSGQFUL should be considered a permanent error, so that NowSMS will not retry submitting the message. If you want to also treat ESME_RTHROTTLED and ESME_RMSGQFUL, it is also possible to include their error codes (58 for ESME_RTHROTTLED and 14 for ESME_RMSGQFUL) in this setting, e.g., "SMPPRejectErrorCodes=All,58,14".


2008-09-16:

* MMSC/MM4: MM4 Acknowledgment responses were returning the header "X-Mms-Request-Status-Code: OK". This has been changed to "X-Mms-Request-Status-Code: Ok" (mixed case "Ok") in order to be consistent with the relevant specification, and to address a real world interoperability issue at a customer site.

2008-09-09:

* SMPP Client: Work-around for delivery receipt processing with an operator SMSC which was returning delivery receipts using the same data_coding value as the original message to which the receipt corresponds, even though the receipt data did not match the coding. (For example, send a unicode message, the data_coding value of the receipt would indicate Unicode, but the actual text of the receipt was a mix of standard text with limited Unicode.) NowSMS now performs additional analysis to allow these receipts to reformat these receipts for correct interpretation.

* SMPP Client: Acknowledge unbind request if received from SMSC, and gracefully terminate connection.

* UCP/EMI Client: Enable translation of delivery receipt reports from UCP/EMI to SMPP format by default, so that applications can request delivery receipts in these environments. If this support needs to be disabled, edit SMSGW.INI, and under the [SMSGW] header, add TrackUCPReceipts=No.

* CIMD Client: Enable translation of delivery receipt reports from CIMD2 to SMPP format by default, so that applications can request delivery receipts in these environments. If this support needs to be disabled, edit SMSGW.INI, and under the [SMSGW] header, add TrackCIMDReceipts=No.


2008-08-26:

* MMSC/MM4: When receiving delivery report (MM4_Delivery_report.REQ) or read reply report (MM4_Read_Reply_report.REQ) transactions over MM4, when an acknowledgment is requested (X-Mms-Ack-Request: Yes), NowSMS would not send the corresponding acknowledgment response (MM4_Delivery_report.RES or MM4_Read_Reply_report.RES) unless a "Sender:" header is present to indicate where the acknowledgment response should be sent. This is all according to the 3GPP TS 23.140 specification, which specifies that "the 'Sender:' header is the system address, to which the corresponding response will be sent." However, in the real world, some MMSCs fail to set the "Sender:" header. This was observed recently with a customer connection to Telenity MMSC. In cases where the "Sender:" header is not present for a delivery report or read reply report where an acknowledgment is requested, NowSMS will now use the SMTP envelope "MAIL FROM:" header as an alternative.

* MMSC: For SMTP E-mail to MMS routing, all text components are now converted to UTF-8 to prevent character set rendering problems.

2008-08-22:

* SMPP: Improve performance of SMPP receipt tracking, which improves sending throughput over SMPP connections when delivery receipts are requested.

* GSM Modem handler update: Recent versions of the Multitech CDMA Modem have a bug in the implementation of the AT+CMGS command, if you experience the error "Error waiting for response from modem (1)", try editing SMSGW.INI, and under the [Modem - driver name] section header, try adding ModemSendWorkaround=Yes. This setting does not apply to the error "Error waiting for response from modem (2)" which is a different type of timeout error condition, from which this update may better recover on problematic modems.


2008-07-25 (only SMSSMPP.DLL updated for this release, version information still says 2008-06-24):

* SMPP: Work-around for strange SMPP SMSC behaviour with long text messages. This particular SMSC expected "Encode long text messages with 7-bit packed encoding" to be UNchecked when a text message is submitted with the default data coding (DCS) value. However, if the data coding specified a message class value, the SMSC would corrupt the message unless "Encode long text messages with 7-bit packed encoding" was checked. (With that setting checked, messages using the default encoding would be corrupt.) As a work-around, if dealing with this situation, edit SMSGW.INI, and under the [SMPP - host:port] section of the file, add GSMPackLongSMSWithMessageClass=Yes.

2008-06-24:

* MMSC: Possible fix for site that periodically experienced MMSC-yyyymmdd.LOG files with only a single entry.

2008-06-20:

* MMSC: MMSRoutingURL callbacks for delivery notifications and read receipts did not always have the "From" field present in the callback.

* MMSC/MM4: For delivery notifications routed via an MM4 connection, the MMSRoutingURL callback was being called multiple times. It is now only called once.

* MMSC/MM4: The message-id returned in delivery receipts and read receipts was not always properly converted to the local message-id before delivery, which could cause problems matching the receipts with the original message.

* MMSC: Versions prior to NowSMS 2006 logged when new user accounts were provisioned automatically to a file named USERDEF.LOG. This logging stopped when the new user database format was introduced in NowSMS 2006. The USERDEF.LOG file has been reenabled.