Concatenated SMS didn't submit

Concatenated SMS didn't submit SearchSearch
Author Message
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 1
Registered: 11-2017
Posted on Friday, December 01, 2017 - 06:27 pm:   

Dear Support,

Kindly i have an issue in long messgaes "concatenated SMS" which are queued and didnt reach the gateway, please advise.
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 2
Registered: 11-2017
Posted on Sunday, December 03, 2017 - 01:10 pm:   

Dears,

Would you please update us in regarding this issue as all concatenated SMSs queued in subfolders under the folder Q and when moving these SMSs under the Q folder directly all SMSs sent on time.
Thanks and waiting for your urgent reply.
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 3
Registered: 11-2017
Posted on Monday, December 04, 2017 - 04:17 pm:   

Dears,

Do you have any update on this issue?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5881
Registered: 08-2008
Posted on Monday, December 04, 2017 - 08:05 pm:   

We need more information.

I'm not aware of any problems like you describe.

What type of SMSC connection?

What version of NowSMS? (If thIs is an older version, there may be fixes in newer versions.)

If the subfolders are causing an issue, they can be disabled ... on the "SMS Users" page, uncheck "separate user queues".

--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 4
Registered: 11-2017
Posted on Monday, December 04, 2017 - 09:25 pm:   

Dear Des,

Thanks for your reply.

I already disabled this option "separate user queues", but I asked for the SMSs which long than one SMS which counted as 2,3 or more than these, like the below:

if I sent SMS long and counted by 2 SMSs, that's mean NowSMS created two .req files, I asked for these files as they created under subdirectory under the Q folder, but I need the NowSMS to create them under the Q folder directly.

Thanks in advance...
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5882
Registered: 08-2008
Posted on Monday, December 04, 2017 - 10:35 pm:   

First, please answer these questions:

What type of SMSC connection?

What version of NowSMS?


quote:

if I sent SMS long and counted by 2 SMSs, that's mean NowSMS created two .req files, I asked for these files as they created under subdirectory under the Q folder, but I need the NowSMS to create them under the Q folder directly.




What subdirectory names is NowSMS using? Please give multiple examples.

What filenames do you see? Again, multiple examples would be helpful.

How are the messages being submitted to NowSMS? HTTP. I'm assuming.

--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 5
Registered: 11-2017
Posted on Tuesday, December 05, 2017 - 08:35 am:   

SMSC Connection:

The client connects to NowSMS by SMPP protocol, then I forward these SMSs to the providers HTTP.

NowSMS version is --> Now SMS/MMS Gateway v2012.06.28

The subdirectory names --> as the below:
1- If the messages are short created under directories with the name like these: ###0001,###0002
2- If the messages are long created under directories with the name like these: ###24F7E,###24F7F,###24F80

All these folders under the Q folder and have .req files, what I need to create all .req files under the Q folder directly.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5883
Registered: 08-2008
Posted on Wednesday, December 06, 2017 - 11:02 am:   

Hi,

OK....this took some deep investigation. I think this setting under [SMSGW] in SMSGW.INI will do what you want:

MaxQDirEntries=4000000000

--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 6
Registered: 11-2017
Posted on Wednesday, December 06, 2017 - 12:04 pm:   

Hi Des,

Perfect, this is working fine, appreciated your efforts.

I have another issue with the long SMS when it is exceeded 6 SMSs, the message exceeded the number of retrials then failed like the below message:

SAR-20xxxxxxxxxx-5a4b18b1-11-1.req
SAR-20xxxxxxxxxx-5a4b18b1-11-2.req
SAR-20xxxxxxxxxx-5a4b18b1-11-3.req
SAR-20xxxxxxxxxx-5a4b18b1-11-4.req
SAR-20xxxxxxxxxx-5a4b18b1-11-5.req
SAR-20xxxxxxxxxx-5a4b18b1-11-6.req
SAR-20xxxxxxxxxx-5a4b18b1-11-7.req
SAR-20xxxxxxxxxx-5a4b18b1-11-8.req
SAR-20xxxxxxxxxx-5a4b18b1-11-9.req
SAR-20xxxxxxxxxx-5a4b18b1-11-10.req
SAR-20xxxxxxxxxx-5a4b18b1-11-11.req

Would you please check and update me.
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 7
Registered: 11-2017
Posted on Thursday, December 07, 2017 - 01:58 pm:   

Hi Des,

Do you have any update?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5884
Registered: 08-2008
Posted on Friday, December 08, 2017 - 09:30 am:   

Again, I don't have enough information.

What errors are reported in the log (SMSOUT-yyyymmdd.LOG) before each retry?

Do all segments fail, or are some sent?

This message is being submitted to NowSMS via SMPP client, correct? Is it using message_payload to submit in one transaction? There is a note in our changes log (https://www.nowsms.com/download/changes.txt) that versions prior to 2013.06.06 have an issue with message_payload submissions longer than 1023 bytes.

--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 8
Registered: 11-2017
Posted on Friday, December 08, 2017 - 02:29 pm:   

Hi Des,

Please find the below reply to each question:

What errors are reported in the log (SMSOUT-yyyymmdd.LOG) before each retry?
[Reply]
2017-12-06 13:52:43,SAR-2xxxxxxxxxxx-5a4b18b1-11-1.req,x.x.x.x,2xxxxxxxxxxx,Retry Pending - HTTPSMSC: Connection failed, HTTP status code = 404. -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-2.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-3.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-4.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-5.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-6.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-7.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-8.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-9.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-10.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx
2017-12-06 15:27:56,SAR-2xxxxxxxxxxx-5a4b18b1-11-11.req,x.x.x.x,2xxxxxxxxxxx,OK -- HTTP - x.x.x.x#108:80,SubmitUser=xxxx;Sender=xxxx

Do all segments fail or are some sent?
[Reply]
The first part exceeds the numbers of retrials then .req file under the Q folder replaced with another file with extension .err

This message is being submitted to NowSMS via SMPP client, correct?
[Reply]
Yes

Is it using message_payload to submit in one transaction?
[Reply]
I didn't think that as the message received in more than one part.
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 9
Registered: 11-2017
Posted on Monday, December 11, 2017 - 12:14 pm:   

Hi Des,

Do you have any update, please ?
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 10
Registered: 11-2017
Posted on Tuesday, December 12, 2017 - 03:32 pm:   

Hi Des,

Would you please update me if you have any comments on this issue.
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 11
Registered: 11-2017
Posted on Thursday, December 14, 2017 - 09:32 am:   

Hi Des,

We need your support on this issue, please.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5886
Registered: 08-2008
Posted on Thursday, December 14, 2017 - 02:54 pm:   

Sorry for the delay.

As this is an HTTP connection, it can be difficult to troubleshoot.

According to your log, the provider is returning an HTTP 404 error (page not found).

My guess is that the provider has a message length limit...most likely a URL length limit.

I just tried a 14 segment message via an HTTP SMSC connection without any problem.

Here's what I suggest for you to troubleshoot...

Enable the SMSDEBUG.LOG, and look for the HTTP URLs that NowSMS generates for transmitting the message to the provider.

Try these URLs manually to determine the max length supported by the provider.

Version 2012.06.28 does have some support for HTTP POST templates, which might be an alternative if the provider issue is an HTTP GET URL length limitation. If you do want to try HTTP POST, I can help create a template, but need details on the provider's HTTP POST API.


--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 12
Registered: 11-2017
Posted on Thursday, December 14, 2017 - 05:26 pm:   

Hi Des,

Thanks for your input, as we took the URL from SMSDEBUG.LOG and hit it manually and give us the same error 404, so we will check with our provider.


Sorry dear for bothering you, but I have one more issue and it is the last one I have, please find the below description:

Some of SMSs which concatenated (more than one part) not submitted from the NowSMS application as the SAR ID not equal to all parts and if I change it manually the message sent successfully, please find the below logs:

SAR-2xxxxxxxxxxx-5a6f712b-3-1
SAR-2xxxxxxxxxxx-5a6f712a-3-2
SAR-2xxxxxxxxxxx-5a6f712a-3-3

As you see the ID of the first part is different than the other parts.

Would you please check this issue and feed me back.
Finally, thank you again for your support.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5887
Registered: 08-2008
Posted on Thursday, December 14, 2017 - 06:41 pm:   

This suggests one of the following scenarios:

* Different segment reference numbers from the submitting SMPP client (sar_msg_ref_num or equivalent parameter in the UDH), which technically makes them separate messages.

* More than 5 minutes elapsed between submission of these segments.

The SMSWEB-yyyymmdd.LOG should report the sequence of submission to determine if the 5 minute scenario is occurring...I assume it is not.

To confirm that the problem is with the SMPP client is difficult. We would need to see a raw SMPP packet trace of the problem messages. (Either a Wireshark trace, or the SMPPDEBUG.LOG that NowSMS generates when SMSDEBUG.LOG is enabled.)



This said, I can see that there is a bug in the version of NowSMS you are running, where messages with missing segments (which these are, assuming they have different reference numbers) get stuck in the queue. I see the following change note in a 2013 update:


quote:

2013-05-31:

* HTTP SMSC: If "Send Long Messages without Segmentation" is enabled for a connection, any segmented messages where a part is missing would get stuck in the queue. With this update, any such stuck messages will get released automatically after 3 minutes.




--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 13
Registered: 11-2017
Posted on Friday, December 15, 2017 - 07:32 pm:   

Hi Des,

Please note that I sent to you an email with subject "Des - Concatenated SMS didn't submit" to "nowsms@nowsms.com" which contain the SMSDEBUG.LOG in order to check further the issue.

On another hand for the SMSs which returned 404, I tried them from the browser directly and that is failed but I try it with POST and the URL submitted successfully and the SMS delivered to the handset, so I need your help as I already changed function Get to POST in some files which named ("Send Binary Message Other.htm", "Send SyncML OTA Settings.htm" & "Send Text Message.htm") under path "C:\Program Files (x86)\NowSMS\html" but with no any change, so please advice if these changes are right or not or if I should change in any other files.

Many thanks dear for your support...
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5888
Registered: 08-2008
Posted on Friday, December 15, 2017 - 10:24 pm:   

Hi,

Regarding the logs, the messages in question were received before the previous restart. Therefore the details are not in the logs. Try to locate another example and send SMSDEBUG.LOG, SMSDEBUG.BAK, SMPPDEBUG.LOG, SMPPDEBUG.BAK.

Regarding HTTP POST, those templates are not involved.

We do not have documentation on this interface.

But, the basic process is to locate the NowSMS directory in program files (or Program Files (x86)), and look for the HTTPSMSC folder. In that folder, you will find a Twilio folder. Create a new folder under HTTPSMSC which will contain templates for your provider.

Files in this folder should include:

sendtext.txt (HTTP POST template) - example:

POST / HTTP/1.1
Content-Type: application/x-www-form-urlencoded

to=@@PhoneNumber@@&text=@@TEXT@@


template.ini (settings)

[Send]
UseKeepAlive=Yes
LongTextAsText=Yes
UnicodeAsText=Yes
Charset=utf-8


Once this folder and files exist, the folder name will appear as a server type in the HTTP SMSC configuration.

All said, 2012 was the first versions to add support for these templates, so I am unsure how much of this functionality exists in that version.

--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 14
Registered: 11-2017
Posted on Saturday, December 16, 2017 - 02:50 pm:   

Hi Des,

Please note that I already sent to you another email with other logs, please check them and get back to me.

Regarding HTTP POST, I created the folder and named it "HTTPSMSC" then create under it another folder and named it "HTTP-POST" and it already showed in Server Type, but I will contact with my HTTP provider in these files, please if you have any recommendation or more samples from these files it will be very useful if you shared it with me.

Thanks again for your time and for your support.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5889
Registered: 08-2008
Posted on Monday, December 18, 2017 - 09:28 pm:   

Hi,

Regarding the processing of SAR-XXXXXXXX-5a2ac5be-2-1.req and SAR-XXXXXXXX-5a2ac548-2-2.req, here is what is happening...

At 16:16:02:215, part 1 of the message is received. SAR-XXXXXXXX-5a2ac548-2-1.req

At 16:16:03:590, part 2 of the message is received. SAR-XXXXXXXX-5a2ac548-2-2.req. This is the last part, and the message is routed to HTTP.

At 16:20:02:372, the client submits what appears to be part 2 again, with the same reference number (0xBE) as the previous message to this recipient. Less than 5 minutes have elapsed, so the message ID is reused. SAR-XXXXXXXX-5a2ac548-2-2.req

At 16:20:05:372, the client submits what appears to be part 1 again, with the same reference number (0xBE) as the previous message to this recipient. Because this is the first part of a message, and we have already seen part 1 with this same reference number, NowSMS assumes the client has reused the 8-bit reference number, and assigns a new message ID. SAR-XXXXXXXX-5a2ac5be-2-1.req


The underlying problem is that the client is submitting the message more than once. This would be ok, but if less than 5 minutes have passed, and the duplicate is submitted out of sequence, the duplicate message gets different IDs assigned to its segments, and in the 2012 version, this duplicate message gets stuck.

I have discussed this with engineering, and we will be changing how the message ID gets reused in an update to the 2017 version.

Other than updating, all I can suggest is investigating why the client is submitting duplicate messages.


--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 15
Registered: 11-2017
Posted on Tuesday, December 19, 2017 - 05:47 pm:   

Hi Des,

Please note that we tried the above configuration but the SMS not submitted from NowSMS and return error 400, you can check the attached configurations and logs.
Thanks and waiting for your feedback.
application/octet-stream
SMSDEBUG.LOG (7.1 k)
application/octet-stream
template.ini (0.1 k)
text/plain
sendtext.txt (0.4 k)
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 16
Registered: 11-2017
Posted on Thursday, December 21, 2017 - 04:19 pm:   

Hi Des,

Do you have any update?
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 17
Registered: 11-2017
Posted on Friday, December 22, 2017 - 06:59 pm:   

Hi Des,

Do you have any update?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5891
Registered: 08-2008
Posted on Friday, December 22, 2017 - 08:39 pm:   

It's difficult to answer, because I have to check to see what is possible in the older version you are running.

Remove Host: and Content-Length headers from the sendtext.txt template...and that will probably fix the 400.

However, there is a good chance that this POST solution will not work for your provider. The GET request uses hex encoding for Unicode messages. The POST solution uses UTF-8 text.


--
Des
NowSMS Support
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 18
Registered: 11-2017
Posted on Monday, December 25, 2017 - 02:39 pm:   

Hi Des,

Please note that I made the above changes and the POST working fine, but I still have two issues as the below:

1- MessageID=@@MessageID@@: didn't binded by its value (received at HTTP as @@MessageID@@).
2- I need to change 2 Parameters for each POST SMSC (Username/Password) for HTTP Provider, as I
have more than one client on the same server.
application/octet-stream
SMSDEBUG.LOG (6.9 k)
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 19
Registered: 11-2017
Posted on Thursday, December 28, 2017 - 05:44 pm:   

Hi Des,

Do you have any update?
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 20
Registered: 11-2017
Posted on Tuesday, January 02, 2018 - 08:57 am:   

Hi Des,

Would you please update me.
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 21
Registered: 11-2017
Posted on Monday, January 08, 2018 - 09:46 am:   

Hi Des,

We need an update for this issue please, thanks for our usual support.
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 22
Registered: 11-2017
Posted on Thursday, January 11, 2018 - 08:13 am:   

Hi Des,

Do you have any update for the above two points in order to proceed from my end for this solution?
Thanks and waiting for your update.
maged mohamed mostafa mohamed
New member
Username: Melbarky

Post Number: 23
Registered: 11-2017
Posted on Tuesday, January 16, 2018 - 03:56 pm:   

Hi Des,

Please, we need your urgent update here.
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 8259
Registered: 10-2002
Posted on Wednesday, January 17, 2018 - 10:44 pm:   

Hi,

Apologies for the long delay in response.

Please understand that you are running an old and outdated version of our software. For prompt and expert support, you should inquire with our sales department about an upgrade.

Des was attempting to provide an option for a solution, however that recommendation was based on what is available in current product versions. I appreciate his actions to always make extra effort for the customer. However, in this case, the POST template was an experimental feature in the old software version.

I have no problem using @@MessageID@@, @@SubmitUser@@ or @@SubmitPassword@@ in POST templates with the current version of NowSMS.

However, I can confirm that these variables do not work in version 2012.06.28.

So, we asked our engineering team to review archived product versions to understand why and determine if there exists a work-around.

As the old version is considered outdated and unsupported, and this POST interface was considered experimental in that version, this investigation was given extremely low priority. I apologize that neither Des nor I communicated this fact.

The good news is that engineering did finally allocate time for this investigation.

In version 2012.06.28, there is a workaround that can enable these parameters.

Based on your earlier example, here is a template that will enable this workaround.

Add an extra HTTP that includes the missing variables, like this:

POST /InsertNowSMSLogs/InsertNowSMSLogs.asmx/InsertLogWithDLR HTTP/1.1
X-NowSMS-Extra: @@MessageID@@; @@SubmitUser@@; @@SubmitPassword@@
Content-Type: application/x-www-form-urlencoded

UserName=xxxx&Password=xxxx&SMSLang=e&SMSText=@@Data@@&SMSSender=@@Sender@@&SMSR eceiver=@@PhoneNumber@@&priority=5&MessageID=@@MessageID@@&NowSmsUser@@SubmitUse r@@&NowSmsPassword=@@SubmitPassword@@&UDH=@@UDH@@&ReceiptRequested=Yes



-bn

Bryce Norwood
Now SMS/MMS Support