TLV in Accounting Callbacks

TLV in Accounting Callbacks SearchSearch
Author Message
ashot shahbazian
Frequent Contributor
Username: Animatele

Post Number: 100
Registered: 06-2004
Posted on Wednesday, March 16, 2011 - 08:57 pm:   

Hi Des,

Is there a way to have NowSMS insert TLV into messages as the tags would return in response to Accounting Callbacks? This is for dynamic message routing, similarly to how you would return SMSCRoute=abcdef.

There should be a possibility to insert more than one tag, for example:

First tag:value may define the first SMSC upstream.
Second tag:value defines the second uplink if the 1st is unavailable; alternatively, if the routes are "chained" and the message failed on the 1st SMSC the 2nd tag would indicate the secondary SMSC for resubmitting it.
Third and fourth can be the destination and the originating networks' MCCMNC, for proper handling of ported numbers by an SMS Hub SMSC upstream. This is a standard GSMA Open Connectivity requirement.
Another tag:value may be destination networks TAP codes, which are required by some SMS Hubs.

If not presently supported, would it be difficult to implement?

Thanks!

Kind regards,
Ashot
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3016
Registered: 08-2008
Posted on Thursday, March 17, 2011 - 12:55 am:   

Hi Ashot,

That's an interesting idea.

Getting the routing capability added was a bit of an architectural hurdle.

But once we got past that, we've recognised that there is a good possibility to allow the callbacks to perform other changes, we just weren't sure what. Adding/modifying TLV is a good idea.

I will start an engineering discussion on this.

--
Des
NowSMS Support
ashot shahbazian
Frequent Contributor
Username: Animatele

Post Number: 101
Registered: 06-2004
Posted on Thursday, March 17, 2011 - 01:24 am:   

Hi Des,

Another benefit of this (as well as SMSCRoute=xxxx) method is that, unlike combinations of conventional rules by destination and source addresses and usernames on SMSC uplinks, routing by TLV does not strain the server's resourses.

We have tested it by adding by external application ~30 different tags in a random fashion, to a fast stream of submit_sm routed to NowSMS SME over several user accounts. Messages then submitted via 30 SMSC uplinks each having one tag:value specified. No additional CPU load was visible during peaks of 300-400 SMS/sec.

Please let me know if you'd like us to detail some specific cases that would benefit from this functionality.


Kind regards,
Ashot
Paul Friar
New member
Username: Paulinho

Post Number: 1
Registered: 03-2011
Posted on Friday, March 18, 2011 - 03:39 pm:   

Hi Des,

To enable mobile messaging inter-working, GSMA Open Connectivity high level requirements dictate the need for transparency of originator / destination addresses. This is facilitated by the insertion of tags to identify the destination and originating network operators.

The SMPP parameters to be utilized for the insertion of OA and DA tags are source_subaddress and dest_subaddress. The format of the tags is a 6 digit code consisting of MCC+MNC. The parameters source_subaddress_id and dest_subaddress_id are not suitable for semantic reasons.

Can provide more information on this case if required.

Kind regards,

Paul
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3026
Registered: 08-2008
Posted on Saturday, March 19, 2011 - 12:17 am:   

Hi Paul,

We're going to investigate allowing the preauth (and the SmsSend accounting) callback add/modify TLV. So hopefully this will allow what you need.

I would be curious to see the document that you refer to...can you point me to it?

Thanks!

--
Des
NowSMS Support
Paul Friar
New member
Username: Paulinho

Post Number: 2
Registered: 03-2011
Posted on Tuesday, April 05, 2011 - 11:18 am:   

Hi Des,

If you drop me an email then I can send you the document.

Thanks,

Paul
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3082
Registered: 08-2008
Posted on Tuesday, April 05, 2011 - 03:56 pm:   

Hi Paul,


Send it to nowsms@nowsms.com, but put Attention: Des in the subject line, so that it gets routed to my attention.

On a separate note, we are investigating this more fully, as another customer has suggested that the accounting callbacks should be allowed to change other parameters, such as source and destination addresses. (Useful for situations where intermediate systems want to add routing prefixes directly to the source/destination addresses.)

--
Des
NowSMS Support
ashot shahbazian
Frequent Contributor
Username: Animatele

Post Number: 102
Registered: 06-2004
Posted on Tuesday, April 05, 2011 - 04:26 pm:   

Hi Des,

Changing (source) addresses via accouting callbacks would also facilitate implementation of a SAT (source address translation) mechanism, whereas the OA-s in messages from a difficult-to access MNO are replaced dynamically with OA-s from an intermediary SCCP-addressable number pool.

In this case the reply SMS are fetched by ourselves, DA-s in them recalculated back to the MNO subscribers' MSISDN and then sent back to the MNO-s SMSC.

The replies typically come as deliver_sm however, in the receiving part of an SMSC connection from NowSMS. If you are to make NowSMS compatible with the SAT scheme it'd require a method of changing DA in the reverse message stream as well. This would not be trivial, or it it could somehow be done with 2-Way SMS functionality ?

Kind regards,
Ashot
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3097
Registered: 08-2008
Posted on Wednesday, April 13, 2011 - 09:23 pm:   

Hi Ashot,

We're still working on this, but the intent is for there to be the ability to implement bidirectional source and destination address translation.

There is an "SMSIN" accounting callback that occurs when a message is received from an upstream SMSC connection. This allows you to change the DA back to the original OA in the type of scenario that you describe.

The plan is to allow the accounting callback to change OA, DA, and TLV parameters via the "SMSSend" and "SMSIN" callbacks.

We are also adding an option to process PHP scripts for accounting callbacks (and 2-way commands) directly, which may help in some scenarios. Originally this was intended to simplify implementation for lower end systems, as many customers get confused when we explain how the scripts need to run under the control of another web server. But this may hold some promise for higher throughput systems as well. We've implemented a FASTCGI interface to manage and interface with a local PHP installation directly, and my tests have shown it to be reasonably fast.

Another week or two, and the OA/DA/TLV replacement capability via the accounting callback should be ready.

--
Des
NowSMS Support
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3118
Registered: 08-2008
Posted on Wednesday, April 20, 2011 - 06:33 pm:   

Hi Ashot,

A follow-up. The 2011.04.19 version at http://www.nowsms.com/download/nowsms20110419.zip supports the ability to modify OA/DA/TLV in the "SMSSend" and "SMSIN" accounting callbacks.

To enable this capability, it is necessary at this time to add SMSAccountingAllowChanges=Yes to the [SMSGW] section of SMSGW.INI. When this setting is present, NowSMS will parse the HTTP response for "SMSSend" and "SMSIN" accounting callbacks looking for "To=", "Sender=", "ServiceType=", "Validity=" and "SMPPOption_xxxx=" settings.

If any of these text strings are present, the value that follows will be used to replace the existing value. (In the case of SMPPOption_xxxx=, a blank value will completely remove the parameter.)

It is recommended that the HTTP response terminate the value with a new line to act as an end of value string delimiter.

--
Des
NowSMS Support
ashot shahbazian
Frequent Contributor
Username: Animatele

Post Number: 103
Registered: 06-2004
Posted on Wednesday, April 20, 2011 - 07:30 pm:   

Thank you Des, this is great!

I've forwarded this to our engineering, so that they test it. It might take some time, but the new routing engine implementation has been prioritized - so I hope we can share some results soon.

Kind regards, Ashot
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 7955
Registered: 10-2002
Posted on Thursday, May 05, 2011 - 04:00 pm:   

This is a follow-up posting to provide some additional information for the topic discussed in this message thread.

The GSM Association document "Open Connectivity SMS Hubbing Architecture IR.75" can be found at the following link: http://www.gsmworld.com/documents/IR7520_unrest.pdf

That document suggests using the SMPP TLV parameters source_subaddress (0x202) and dest_subaddress (0x203) for encoding routing information for the MCC (mobile country code) and MNC (mobile network code) of the source (sender) and destination (recipient) addresses.

This is done to help simplify the routing of messages by intermediary hubs.

Historically, SMS service providers have defined their own proprietary SMPP TLV to encode this information (if it is made available at all). For example, mBlox defines an mblox_operator parameter using TLV parameter number 0x1402.

Hopefully more providers will use the GSM Association Recommendation in IR.75 in place of proprietary parameters, at least at the basic level. (Additional custom and proprietary tags may be required for more advanced requirements, such as those described in the first message of this thread.)

Future versions of NowSMS (releases after 2011.05.05) will include automatically include support for forwarding these parameters recommended by the GSM Association.

Support can be enabled in prior versions of NowSMS by creating or editing an [SMPPOptions] section in SMSGW.INI and adding the following entries to this section:

[SMPPOptions]
source_subaddress=202,HexString
dest_subaddress=203,HexString

(Note: Support for the HexString parameter format was added in 2007.04.)

It is necessary to use the HexString encoding format for this parameter, because the SMPP specification defines first byte of the value as being a binary code value for the type of subaddress. In the case of the GSM Association values, this first byte is always A0 (hex), indicating user specified.

The next 3 bytes are the MCC, followed by 3 bytes for the MNC.

ASCII character values are used for the MCC and MNC, and then converted to a hex string.

The IR.75 document provides an example using the MCC and MNC for AT&T USA, where MCC=310 and MNC=380.

Converting the MCC value from ASCII to its hex string encoding would produce the following sequence of bytes: 33 31 30, and MNC would result in the following: 33 38 30.

The resulting parameter value in hex string format would be A0333130333830.

To specify this value as a source subaddress in NowSMS via HTTP, the following parameter value would be used:

SMPPOption_source_subaddress=A0333130333830

This thread discusses the use of NowSMS accounting callbacks, which can be used to query and/or update these parameters. (Update support was added in the 2011.04.17 release.)

Most NowSMS users will have no reason to be concerned with these parameters.

However, here are a few of the scenarios where it may be necessary to use these parameters.

1.) 2-way (interactive) SMS applications may receive the SMPPOption_source_subaddress parameter to indicate the mobile network from which the request originated, and be required to supply this value as the SMPPOption_dest_subaddress when generating a reply.

Future versions of NowSMS (releases after 2011.05.05) will apply this logic automatically when processing a 2-way command that returns its text response directly. Other 2-way commands would need to look for the "&SMPPOption_source_subaddress=" URL parameter and generate an appropriate "&SMPPOption_dest_subaddress=" URL parameter when submitting a message reply back to NowSMS.

2.) When NowSMS is being used to route messages between providers, it may be desirable to route messages based upon the MCC/MNC value.

This can be done via SMS accounting callbacks. This concept is described in the following article:

http://www.nowsms.com/dynamic-sms-message-routing-with-http-callbacks

What is not mentioned in that article is that beginning with release 2010.05.17, NowSMS also includes SMPP TLV parameters in the accounting callbacks.

This allows the callback processor to inspect the "&SMPPOption_source_subaddress=" and "&SMPPOption_dest_subaddress=" parameters, using those values to determine which route to use for sending the message. (The callback returns a response of SMSCRoute=xxxxx to specify the route to be used, as described in the link referenced above.)

3.) As an extension to the previous scenario, it may be necessary for a system providing SMS hubbing services to apply or insert the above values.

For example, it may be desirable to apply a source_subaddress automatically to all messages received from a particular connection.

Or it may be desirable to perform HLR lookups and apply dest_subaddress parameters before routing.

NowSMS 2011.04.17 and later allow these TLV parameters to be updated by an accounting callback.

In addition to passing these parameters to accounting callbacks via HTTP URL parameters, NowSMS checks the response from the accounting callback to allow these parameters to be set or updated.

The accounting callback should return a "text/plain" response with any parameters to be updated separated by line breaks.

For example:

SMSCRoute=connectionA
SMPPOption_source_subaddress=A0333130333830
SMPPOption_dest_subaddress=A0323334303135

Note: To enable the ability to override SMPPOption parameter values in the accounting callback response, it is necessary at this time to add SMSAccountingAllowChanges=Yes to the [SMSGW] section of SMSGW.INI.

One other note for those not familiar with the NowSMS concept of accounting callbacks. These are HTTP callbacks where NowSMS send an HTTP request to a URL that you define.

These callbacks would normally reside on a separate web server. However, newer versions of NowSMS also support the ability to host PHP scripts locally, as described in the following article: http://www.nowsms.com/now-sms-native-php-scripts
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3280
Registered: 08-2008
Posted on Tuesday, June 21, 2011 - 05:35 pm:   

Follow-up.

Support was added to automatically support the source_subaddress and dest_subaddress TLV parameters described in this message thread, without requiring them to be explicitly defined in the [SMPPOptions] section.

The initial release with this support (2011.05.23) was buggy. Version 2011.06.20 or later is recommended for this support.

An update is available at the following link: http://www.nowsms.com/download/nowsms20110620.zip

A review of changes in this release is included in the ZIP and is also available separately at http://www.nowsms.com/download/changes.txt

--
Des
NowSMS Support
ashot shahbazian
Frequent Contributor
Username: Animatele

Post Number: 104
Registered: 06-2004
Posted on Thursday, June 23, 2011 - 02:50 am:   

> Thank you Des, this is great! > > I've forwarded this to our engineering, so that they test it. It might take some time, but the new routing engine implementation has been prioritized - so I hope we can share some results soon. > > Kind regards, > Ashot
ashot shahbazian
Frequent Contributor
Username: Animatele

Post Number: 105
Registered: 06-2004
Posted on Thursday, June 23, 2011 - 03:00 am:   

> Thank you Des, this is great! > > I've forwarded this to our engineering, so that they test it. It might take some time, but the new routing engine implementation has been prioritized - so I hope we can share some results soon. > > Kind regards, > Ashot
ashot shahbazian
Frequent Contributor
Username: Animatele

Post Number: 106
Registered: 06-2004
Posted on Thursday, June 23, 2011 - 03:10 am:   

> Thank you Des, this is great! > > I've forwarded this to our engineering, so that they test it. It might take some time, but the new routing engine implementation has been prioritized - so I hope we can share some results soon. > > Kind regards, > Ashot