Accounting Callbacks

Accounting Callbacks SearchSearch
Author Message
Rik Clews
New member
Username: Rikclews

Post Number: 7
Registered: 08-2006
Posted on Tuesday, February 08, 2011 - 02:31 pm:   

Hi,

In order that we can open SMPP to our customers (clients currently connect through http) I've implemented smpp accounting callbacks for routing and pre auth, but have hit an issue in that the receipts of these messages are not passed back through the current 2-way command, unless I uncheck the "enable SMPP login" for the user, which then means they can't bind to us through SMPP.

Can we set it to use the 2-way simulaneously with the http connections?

Alternatively I considered using the SMSIN callback to handle all inbound, but I can find no documentation for this - and my limited testing is showing a lot of missing information, for example no SMPPOption parameters on inbound messages and no smscRoute (only smscName) for receipts and messages.

Ideally we'd like to be able to use callbacks rather than 2 way, keeping all the processing in one place, but certainly only want one or the other.

Any pointers greatly appreciated!

Regards,

Rik
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2888
Registered: 08-2008
Posted on Tuesday, February 08, 2011 - 09:49 pm:   

Hi Rik,

I understand the dilemma.

One of our goals in the SMPP environment is to make sure that delivery receipts get routed back to the submitting account and that the message IDs match up.

Any messages routed to an SMPP user account (including these delivery receipts) do not get routed to the 2-way commands. The messages routed to the 2-way commands are otherwise unroutable inbound messages.

This often causes concern for situations like you describe, where your overall system also wants to see the delivery receipts.

The "SMSIN" callback should provide the information that you need. (It is called for any messages received from an outbound SMSC connection.) We've considered moving the delivery receipts to a separate callback, but people are using this one already and we don't want to cause confusion.

You are correct that it includes the SMSCName parameter (e.g., server:port), but not SMSCRoute. The reason for this is because the route name (which can be shared by multiple connections). The reason for this is because we were originally associating that route name with outbound messages only ... but I know we do have an option to pass that name to 2-way commands.

So ... famous last words ... we could probably add the route name to the "SMSIN" accounting callback quite easily, if you absolutely need it.

Regarding SMPPOptions, they should be present in the "SMSIN" callback. Are you certain that they are not present? (Maybe there just aren't any in the delivery receipts?)

--
Des
NowSMS Support
Rik Clews
New member
Username: Rikclews

Post Number: 8
Registered: 08-2006
Posted on Wednesday, February 09, 2011 - 03:16 pm:   

Any chance of publishing some info on the SMSIN and SMSOUT callbacks? Would help a lot to know the possible parameters.

I'm testing now but do you know if I can refuse an inbound message or receipt (for example if we have a database error) in such a way that they will be retried, by either NowSMS or SMSC? Along the lines of the "SMPPErrorCode=0x66" I'm thinking. Its just that the 2 way will preserve the messages as the BAD files which we can reprocess, would hate to lose messages for a local network issue.

Is there a chance that the callback can give us similar info to the 2 way? I mentioned the route name but there appears to be no message ID which we use to guard against duplicates, or message date/time which can show us delays through the system.

As for the SMPPOptions, thats my fault. I was testing with an SMPP simulator, just the test system uses the live route names but the sim doesn't give the options - doh!
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2892
Registered: 08-2008
Posted on Wednesday, February 09, 2011 - 10:24 pm:   

Hi Rik,

I will make an attempt to document these callbacks.

However, before I do ... let me explain that the reason there is no "MessageID" parameter is because the callback occurs before one is generated.

Unfortunately, the callback cannot block/refuse an inbound message.

The reason the callback occurs before the MessageID is assigned, however, is because we were allowing for the potential future possibility of allowing the callback to block the message ... either to discard without processing, or to return an error code back to the other side.

Let me emphasize that NowSMS does not currently support this.

Unlike 2-way commands, accounting callbacks are not retried.

Here is an attempt at documenting these callbacks. It is possible that I may have missed a detail or two.

SMSIn Accounting Callback

This callback is executed when an SMS message is received from an upstream SMSC connection.

The response to this HTTP callback is currently ignored. A standard HTTP 200 OK response is encouraged for future compatibility.

The following variables can be set for the accounting callback:

Type=SMSIN
To=Message Recipient Phone Number
Sender=xxxxxx
Binary=1 (optional, will be present if the message is binary)
PID=# (optional, will be present only if a non-zero PID value was specified in message submission)
DCS=# (optional, will be present only if a non-zero DCS value was specified in message submission)
UDH=HexString (optional, will be present only if message contains User Data Header)
Data=HexString (optional, will be present only if message is binary)
Text=String (optional, will be present only if message is text)
SMSCReceiptMsgID=String (optional, NowSMS assigned message ID will be present if this is a delivery receipt)
SMSCReceiptMsgIDOrig=String (optional, upstream SMSC assigned message ID will be present if this is a delivery receipt)
SMSCName=String (the SMSC connection from which this message was received in the format that it appears in the NowSMS SMSC list, e.g., "SMPP - servername:port")


Any defined SMPPOption parameters will also be included.


SMSOut Accounting Callback

This callback is executed when a submitted SMS message is dispatched to an upstream SMSC connection, or queued for a local SMPP user account.

The response to this HTTP callback is currently ignored. A standard HTTP 200 OK response is encouraged for future compatibility.

The following variables can be set for the accounting callback:

Type=SMSOut
From=String (local user account name, or upstream SMSC connection name)
To=Message Recipient Phone Number
MessageID=Message ID assigned to the message by NowSMS
SubmitIP=a.b.c.d (not present for messages received from upstream SMSC connection)
Sender=xxxxxx
Binary=1 (optional, will be present if the message is binary)
PID=# (optional, will be present only if a non-zero PID value was specified in message submission)
DCS=# (optional, will be present only if a non-zero DCS value was specified in message submission)
UDH=HexString (optional, will be present only if message contains User Data Header)
Data=HexString (optional, will be present only if message is binary)
Text=String (optional, will be present only if message is text)
MessageID=String (NowSMS assigned message ID)
SMSCMsgId=String (upstream SMSC assigned message ID, if available)
SMSCName=String (the SMSC connection to which this message was routed in the format that it appears in the NowSMS SMSC list, e.g., "SMPP - servername:port")
Status=String (Starts with "OK", "Retry Pending" or "ERROR" to indicate message disposition)


Any defined SMPPOption parameters will also be included.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2893
Registered: 08-2008
Posted on Wednesday, February 09, 2011 - 10:32 pm:   

All this said. I think we need to make some changes to meet your requirements. I'm just not sure what those changes are.

Adding the route name associated with the SMSC connection should be a simple change.

But maybe the best solution would be for us would be to add an option to copy all delivery receipts being routed to a local user account so that they are also routed to the 2-way command processing queue.

It would probably be easier for us to make that change than to modify the accounting callbacks drastically (especially with regard to adding the option to reject messages). However, I need to review that with our engineering team.

Any additional thoughts?

--
Des
NowSMS Support
Rik Clews
New member
Username: Rikclews

Post Number: 9
Registered: 08-2006
Posted on Thursday, February 10, 2011 - 12:49 pm:   

Thanks for the response.

One thing I found strange was it seemed inconsistent that the SMSIN and SMSOUT included all traffic, though the 2 way excluded SMPP users from the inbound.

I think if we could have SMPP users included in the 2 way it’d help. Also a configuration so we can turn callback features off, we are approaching 20 million messages a month and if we use callbacks for SMSSEND, and 2 way for inbound, it’d be nice to avoid 20 million unnecessary SMSIN calls (same goes for SMSOUT), if we are not using them.

Could you let us know if and when these changes could be made? I don’t think I can proceed with my current task until the 2 way inclusion is available.

But it would also be good to have some consistency between callbacks and 2 way, having SMSIN act in a similar manner though that sounds like a large job.

Many thanks,

Rik
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2898
Registered: 08-2008
Posted on Thursday, February 10, 2011 - 09:31 pm:   

Hi Rik,

I think the issue is that the 2-way command purpose and the accounting callback purpose are different.

It just so happens that they can both be used to process delivery reports ... and neither interface was originally designed for such.

I've talked to our engineering guys, and they agree that a configuration setting to duplicate a receipt being routed to a local SMPP user account so that it also gets routed to the 2-way command processing, is relatively easy to implement. I am supposed to receive an update to start testing tomorrow. It's difficult to estimate when this will be ready to be sent out. (We have been asked for this capability quite a few times in the past.)

Regarding disabling those accounting callbacks, there is an existing setting that can be used to disable the SMSIN and SMSOUT callbacks. Add AccountingExtended=No under the [SMSGW] header or SMSGW.INI.

--
Des
NowSMS Support
Rik Clews
New member
Username: Rikclews

Post Number: 10
Registered: 08-2006
Posted on Friday, February 11, 2011 - 08:38 am:   

Thats excellent Des, many thanks - I'll await word of the update!
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2917
Registered: 08-2008
Posted on Wednesday, February 16, 2011 - 10:50 pm:   

Hi Rik,

Follow-up.

A configuration option has been added for the [SMSGW] section of SMSGW.INI:

DuplicateUserReceiptsFor2Way=Yes

When this configuration setting is in place, if an inbound receipt is being routed to a local SMPP user account, it is also copied to the 2-way SMS queue.

This setting has no effect on other messages, only inbound receipts getting routed to a local SMPP user account.

The update link is at http://www.nowsms.com/download/nowsms20110216.zip.

--
Des
NowSMS Support
Rik Clews
New member
Username: Rikclews

Post Number: 11
Registered: 08-2006
Posted on Friday, February 18, 2011 - 02:57 pm:   

That seems to be working just fine. Many thanks for the swift reaction.

Would it be possible to have the callbacks updated too though? As mentioned, our ideal setup would be to move to an all callback scenario rather than a mix of callback and 2 way. We do have plans to use the SMSOUT callback to help track our message movement to and from the NowSMS gateway which leaves us with the wasted calls from SMSIN which we could be making use of to save making the 2 way calls.

Mainly I'm thinking of retries on SMSIN and the route name parameter.

In the meantime we're up and can expose SMPP and http together for the first time, so thanks for that.

Cheers,

Rik
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2927
Registered: 08-2008
Posted on Monday, February 21, 2011 - 02:15 pm:   

Hi Rik,

I'm trying to re-analyze this to see what the original issue you had with the "SMSIN" callbacks were.

The way I see it, the main problem is that the "SMSIN" callback (and "SMSOUT" for that matter) does not include the "SMSCRoute=" parameter, only "SMSCName=".

The lack of a message ID was also originally cited as an issue, but I'm not sure that it is. The 2-way queue is an actual message queue where messages are stored in a queue and processed. The reason for the presence of a message ID is to detect a situation where there is a problem calling the 2-way command and preventing duplicate message processing. By contrast, the accounting callbacks are processed in real time, there are no retries, and there is no message ID included because there is no concept of a queue being processed here.

As I understand it, getting "SMSCRoute=" added to the "SMSIN" and "SMSOUT" callbacks should not be a major issue.

Am I missing something?

--
Des
NowSMS Support
Rik Clews
New member
Username: Rikclews

Post Number: 12
Registered: 08-2006
Posted on Wednesday, February 23, 2011 - 09:24 am:   

Hi Des,

It was the retry that was our major stumbling block.

The thing with the callbacks was we couldn't guarantee acknowledgement of a message - for example we rebuild the database indexes at 2am and this has been known to cause timeouts. Should a message arrive when this is happening, we could lose it and never know. So the retry of inbound receipts or messages is vital to us.

Route name would help tho!

Cheers,

Rik
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2945
Registered: 08-2008
Posted on Wednesday, February 23, 2011 - 07:16 pm:   

Hi Rik,

Ok ... that makes sense. It was your concern about retries that led me to conclude that making a change to allow the 2-way command processing would offer you a better solution than adding the additional parameters to the accounting callback.

Unfortunately, we can't queue the accounting callbacks. The intent is that the log files act as a backup for the accounting callbacks if necessary.

Although, that said, there are some inconsistencies, for example, SMPPOptions is in the "SMSIN" accounting callback, but not the log entry. And the lack of any message ID prevents the possibility of correlating the callbacks with log entries to detect callbacks that were lost due to timeouts. So obviously this strategy needs some further analysis.

--
Des
NowSMS Support
Rik Clews
New member
Username: Rikclews

Post Number: 13
Registered: 08-2006
Posted on Thursday, February 24, 2011 - 09:08 am:   

The 2 way is perfect for our needs, but we'd like to make use of the SMSOUT callback so we can track the fact that the messages have been submitted and are no longer within our control. That leaves us with the SMSIN calls which we currently can't use which seems a waste.

But a quick fix could be to extend the "AccountingExtended=No" ini switch to have the ability to turn off/on the SMSIN and SMSOUT callbacks individually(?)