SMS - Binary - file2hex

SMS - Binary - file2hex SearchSearch
Author Message
Larry Leung
New member
Username: Lchl888

Post Number: 1
Registered: 08-2006
Posted on Thursday, August 17, 2006 - 06:54 am:   

Hi Bruce,

I notice when I use the NowSMS file2hex.exe to convert a file to hex the size of the file / message doubles in size ?

I have a J2ME application which listens out on port:5000 for a binary SMS message, in particular a gzip binary file. Once it's received, the J2ME application uncompresses the gzip binary back into text.For example, a 320 characters or 320 bytes can be compressed to about 140 bytes.

This works fine with the J2ME application on two cell phones sending and receiving a binary gzip file over SMS.

Now I am trying to send the gzip file as a binary SMS on NowSMS. I convert the file to hex using file2hex.exe. The resulting hex file is now double the size (bytes) of the original gzip file ? i.e instead off being ONE SMS message of 140 bytes, it ends up being two SMS messages ?

The setting I am using are:

NowSMS SMS Binary:
UDH: 06050413880000 (port:5000)
Binary Data: 5468697320697320612074657374206D657.... (for example)
PID:0
DCS:0

What am I doing wrong ?

Thanks in advance for your help.

Larry
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 6370
Registered: 10-2002
Posted on Thursday, August 17, 2006 - 05:38 pm:   

Hi Larry,

When you convert the file to a hex string, it is going to double in size. That is normal.

But then when you actually send the message out, that hex string is going to be converted back into binary where each 2 characters become a single binary byte.

I'd probably have to see the exact raw data that you're sending in order to comment.

But if your file is exactly 140 bytes, then you're going to have a problem.

The UDH header for the port number counts as part of the 140 byte binary limit. So that's 7 bytes of overhead, meaning that you only have 133 bytes (266 characters in a hex string) that can fit into a single message.

-bn
Larry Leung
New member
Username: Lchl888

Post Number: 2
Registered: 08-2006
Posted on Friday, August 18, 2006 - 01:14 pm:   

Hi Bryce,

Thanks for your prompt reply. Your reply makes a lot of sense. After a little testing I notice the following... using exactly 152 characters which is the maximum number of characters for a SINGLE SMS message using binary SMS i.e.:

Using SMS Binary:
UDH: 06050413880000 (port:5000)
Binary Data:The Now SMS/MMS Gateway (NowSMS) is an SMS and MMS Content Delivery Solution. NowSMS is a fast track to deploying and developing SMS, MMS and WAP Push**
PID:0
DCS:0

This goes through as a single SMS. Now, converting the same message into HEX gives:

Using SMS Binary:
UDH: 06050413880000 (port:5000)
Binary Data:546865204E6F7720534D532F4D4D53204761746577617920284E6F77534D532920697320616 E20534D5320616E64204D4D5320436F6E74656E742044656C697665727920536F6C7574696F6E2E2 04E6F77534D532069732061206661737420747261636B20746F206465706C6F79696E6720616E642 0646576656C6F70696E6720534D532C204D4D5320616E642057415020507573682A2A
PID:0
DCS:0

I would have expected this to also go out as a SINGLE SMS but I get TWO SMS sent out ? Did I miss something ?

MessageID=SAR-4081111111-b9-2-1.req, Recipient=4081111111
MessageID=SAR-4081111111-b9-2-2.req, Recipient=4081111111

Regards,

Larry
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 6385
Registered: 10-2002
Posted on Friday, August 18, 2006 - 06:24 pm:   

Hi Larry,

When you also talk about sending regular text messages, things get a bit more complicated.

Let me try to explain.

In GSM environments, the size limit for a single SMS message is 140 bytes. This size limit includes all message data and the user data header (UDH).

As you are probably aware, the size limit for a text SMS message is 160 characters.

This discrepancy is because the GSM SMS character set is a 7-bit character set (see http://support.nowsms.com/discus/messages/1/1103.html). When a text message is sent out, 8 7-bit text characters get packed into 7 8-bit binary bytes. So 160 7-bit text characters can be packed into 140 8-bit binary bytes.

This can get a bit more confusing, as there is also an extended GSM character set with some characters like "{", "}", "[", "]", "€" and a few others are represented as two 7-bit characters.

And if you want to include any characters not in the GSM 7-bit character set in your message, then the entire message needs to be encoded in 16-bit Unicode, where only 70 16-bit Unicode characters can fit into 140 8-bit binary bytes of an SMS message.

Anyway ... how does this relate to your above examples and questions?

When you submit a binary SMS message to NowSMS, we scan the data string to determine if it is a hex string or if it is regular text.

If it is regular text, then we perform the 7-bit packing of the text. That is why you encountered a 152 character limit above. Factoring out the UDH, we were able to fit 152 7-bit text characters into the remainder of the 140 byte message.

If you just convert the text characters to a binary hex string, then we are going to send that out as the raw hex. And the resulting message is likely to be corrupt, unless you set DCS=4 or F5 to indicate that the message should be treated as a binary. (DCS=0 indicates that the message is a text message, and it would be assumed that any text is already 7-bit packed.)

Hopefully that will clarify more than it confuses.

If we go back to your original plan ... compacting text with gzip ... then you would truly have 8-bit binary data that you would be sending ... and it could not (and should not) be 7-bit packed. It also should be sent with a DCS value of 4 or F5 to indicate that the content is binary. (We usually recommend using a value of F5, but sometimes in the USA you have to use a value of 4.)

-bn