HTTP Not Found when retrieving MMS from campaign

HTTP Not Found when retrieving MMS from campaign SearchSearch
Author Message
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 23
Registered: 06-2010
Posted on Friday, July 23, 2010 - 05:13 pm:   

Hi Des,

I'm in a middle of a MMS campaing, and there are a lot of messages that failed with this log entry (MMSC-20100723.LOG):
2010-07-23 13:01:36,MMSRETRIEVE,200.40.175.196,,+59898100658,20100723/09/E389D7D1.MMS,FAILE D

when correct messages are logged like this:
2010-07-23 13:02:53,MMSRETRIEVE,200.40.175.196,23456,+59891296073,20100723/09/E389D7D1.MMS, OK,25655

I've traced failed messages with Wireshark, and they appear when NowSMS respond with "HTTP Not Found"

For instance:

GET /20100723/09/E389D7D1.MMS?id=%2b59898035770 HTTP/1.1

Host: 192.168.30.22

X-Hts_apn: mms

Accept: */*, text/html, text/vnd.wap.wml, text/plain, text/x-vcalendar, text/x-vcard, text/x-imelody, text/vnd.wap.wmlscript, text/calendar, text/css, text/vnd.wap.sl, text/vnd.wap.si, text/vnd.wap.co, text/vnd.wap.connectivity-xml, image/jpeg, image/gif, image/bmp, image/vnd.wap.wbmp, audio/imelody, audio/midi, audio/x-midi, audio/mid, audio/wav, audio/x-wav, audio/sp-midi, audio/amr, multipart/mixed, application/vnd.wap.wmlc, application/vnd.wap.wmlscriptc, application/vnd.wap.multipart.related, application/vnd.wap.multipart.mixed, application/vnd.oma.drm.message, application/vnd.oma.drm.content, application/vnd.oma.drm.rights+xml, application/vnd.oma.drm.rights+wbxml, application/vnd.wap.xhtml+xml, application/vnd.wap.hashed-certificate, application/vnd.wap.signed-certificate, application/vnd.wap.wtls-ca-certifcate, application/x-x509-ca-cert, application/smil, application/vnd.oma.dd+xml, application/xhtml+xml; profile="http://www.wapforum.org/xhtml", application/vnd.wap.connectivity-wbxml, application/vnd.wap.mms-message, application/xhtml+xml, application/vnd.wap.wbxml, application/vnd.wap.slc, application/vnd.wap.sic, application/vnd.wap.coc, */*

Pragma: no-cache

Cache-Control: no-cache

X-Hts_clid: +59898035770

X-Wap-Profile: "http://www-ccpp.tcl-ta.com/files/ALCATEL-OT-303.rdf"

Accept-Charset: utf-8, utf-16, iso-8859-1, iso-8859-2, ISO-8859-15, iso-10646-ucs-2, us-ascii

User-Agent: Alcatel-OT-303/1.0 ObigoInternetBrowser/Q03C

Max-Forwards: 10

X-Forwarded-For: 200.40.246.18

X-Forwarded-Host: s4.mms.ancel.net.uy

X-Forwarded-Server: s4.mms.ancel.net.uy



HTTP/1.1 404 Not Found

Content-Length: 0

Connection: Keep-Alive

-------
While running this campaing in a Xeon dual core processor 2Gb RAM, I get 100% cpu processing sometimes (not for too long) and I'm processing between 100 and 200 simultaneous connections.

Do you know what could be happening? It seems to be something with NowSMS application, because same message is correctly retrieved by other clientes.

Thank you,
Gonzalo
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2353
Registered: 08-2008
Posted on Friday, July 23, 2010 - 05:25 pm:   

Hi Gonzalo,

If the message ID is the same, this would indicate that the recipient (phone number after id=) is not found in the recipient list of the message.

There should be a file MMSCDATA/20100723/09/E389D7D1.INI

Under the [Recipient Status] header, there is a list of all recipients.

Is +59898100658 in this list?

How many recipients was the message broadcast to? I am wondering if maybe there is a problem doing this recipient check if the recipient list is very large.

If +59898100658 is in that list, as a temporary fix, add the following setting to the [MMSC] section of MMSC.INI:

MSISDNHeaderRequiredForReceive=No

The MMSC service must be restarted to notice a change to this particular parameter setting.

If the phone number is in the list, I would be curious to see this particular INI file to understand if we are encountering some sort of problem with large recipient lists.

--
Des
NowSMS Support
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 24
Registered: 06-2010
Posted on Friday, July 23, 2010 - 05:40 pm:   

Hi Des,

thank you for your quick answer.
The mentioned number is indeed in the list:
+59898100658=Delivered
and I have configured MMSC.INI with the key:
MSISDNHeaderRequiredForReceive=NO

This campaing is 36000 clientes long, and was started by the web interface.

I'm attaching current message .INI (still half an hour to finish campaing -more 5k queued SMS messages)

Thank you again,
Gonzalo

ps. that message was now retrieved ;-), maybe due to another retry by the terminal
application/zipE389D7D1.INI file
E389D7D1.zip (552.8 k)
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2354
Registered: 08-2008
Posted on Friday, July 23, 2010 - 05:48 pm:   

What version of Windows is the server running?
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 25
Registered: 06-2010
Posted on Friday, July 23, 2010 - 06:33 pm:   

Is 2003 Std. Server.

I will try to start a debug..

Thanks,
Gonzalo
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 26
Registered: 06-2010
Posted on Friday, July 23, 2010 - 06:37 pm:   

Another tip: It only happens while campaign is being performed (after the queue is empty, messages are being correctly retrieved).

Best regards,
Gonzalo
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2356
Registered: 08-2008
Posted on Friday, July 23, 2010 - 06:45 pm:   

Hi Gonzalo,

Win 2003 should be ok. I initially thought there might be platform versions differences, but I don't think there are.

I suspect there is a file locking issue/problem with a large number of recipients on the same message ID. However, I'm going to need our engineering team to verify this. (I'm still working on a scenario to recreate the problem.)

MSISDNHeaderRequiredForReceive=No will provide a work-around if the problem is what I suspect.

The only problem is that some recipient statuses may not be properly updated to "Delivered" which could trigger unnecessary retries (or conversion to SMS with web link, which I expect you have already disabled for these messages).

--
Des
NowSMS Support
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2357
Registered: 08-2008
Posted on Friday, July 23, 2010 - 06:51 pm:   

Hi Gonzalo,


quote:

Another tip:...




Ah ... that latest clue makes perfect sense.

The problem is not in the area that I was suspecting.

The problem is that this ".INI" file is not written out until all recipients have been processed.

MSISDNHeaderRequiredForReceive=No will still function as a work-around. But the problem remains that recipient statuses are not properly updated to "Delivered" during this phase.

This should be easy to recreate. Hopefully it is easy to fix.

--
Des
NowSMS Support
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 27
Registered: 06-2010
Posted on Friday, July 23, 2010 - 09:29 pm:   

Hi Des..

Just a thought.. if you're running two similar processes (it's a dual core system), could it be possible that both try to access same file simultaneously then having this file lock problem?

Regards,
Gonzalo
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2363
Registered: 08-2008
Posted on Sunday, July 25, 2010 - 05:54 pm:   

Hi Gonzalo,

The issue is that the recipient information is not recorded in that INI file until after all recipients have been processed/queued.

For a large broadcast, initial recipients may receive the notification before the recipient information is recorded. This causes the MSISDNHeaderRequiredForReceive check to fail.

We are investigating ways to address this in a future update.

--
Des
NowSMS Support
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 28
Registered: 06-2010
Posted on Monday, July 26, 2010 - 02:10 pm:   

Hi Des,

just for the records, let me tell you that even I have configured MSISDNHeaderRequiredForReceive=No, the problem still appears.
Some informational numbers: in a 2.5MMS/sec system, with 36k distribution lists, about 6k messages fail and 4k of these are retrieved later.

I'll be running this campaing throughout the whole week (4hours a day)

Thank you,
Gonzalo
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2365
Registered: 08-2008
Posted on Monday, July 26, 2010 - 08:03 pm:   

Hi Gonzalo,

I've completed more testing, and the MSISDNHeaderRequiredForReceive=No has no effect. (This setting is actually off by default and is for a different purpose.)

The correct setting that will work around this problem is MSISDNMatchRequiredForReceive=No.

I was confused between two different infrequently used settings with similar names.

I will post more details when we have a fix, but I do believe we have identified the problem, and this corrected setting will provide a work-around until a fix is available.

--
Des
NowSMS Support
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 29
Registered: 06-2010
Posted on Monday, July 26, 2010 - 08:57 pm:   

Hi Des,

I *really* appreciate your help :-). I will try this setting during our next 36k clients campaing tomorrow and let you know the results!.

Thank you again.
Regards,
Gonzalo
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 30
Registered: 06-2010
Posted on Tuesday, July 27, 2010 - 07:50 pm:   

Hi Des,

Just to let you know that the setting you've told me worked fine.

Regards,
Gonzalo

ps. might it be possible that it even made the system to performing better?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2372
Registered: 08-2008
Posted on Tuesday, July 27, 2010 - 09:51 pm:   

Hi Gonzalo,

This setting would improve performance for a scenario of a single message addressed to a large number of recipients.

It would not have any impact on normal MMS message traffic.

What performance differences did you notice in your sending?

--
Des
NowSMS Support
Gonzalo Escuder
New member
Username: Gdescuder

Post Number: 31
Registered: 06-2010
Posted on Wednesday, July 28, 2010 - 05:59 pm:   

This is exactly the case ;-)..
What I've seen is that the average number of simultaneous tcp sessions (retrieving messages) have lowered to less than 100 (there were between 100 and 200 before)..


Regards,
Gonzalo