Conversion of Binary to text

Conversion of Binary to text SearchSearch
Author Message
Shailendra Bansal
New member
Username: Embarc

Post Number: 1
Registered: 01-2007
Posted on Monday, January 08, 2007 - 04:16 pm:   

I am working on a vehicle tracking application wherein the vehicle unit sends location data as sms in binary format. How to convert the received sms to text format?
Malcolm - Now Support
Moderator
Username: Malcolm

Post Number: 75
Registered: 12-2006
Posted on Monday, January 08, 2007 - 08:12 pm:   

Hi,

If the data is in a binary format, then it is a binary format that is specific to the application that is sending it.

In order to understand and interpret the format, you either need documentation on the binary format that the application is using ... or you need to reverse engineer it.

You can, of course, receive the binary format SMS message with NowSMS, but NowSMS will only see it as a string of binary bytes. You need to know the format of the binary data in order to understand and translate the binary data into a format that is meaningful to you.

--
Malcolm
Now Wireless Support
Shailendra Bansal
New member
Username: Embarc

Post Number: 2
Registered: 01-2007
Posted on Tuesday, January 09, 2007 - 05:30 am:   

The data when received from the vehicle unit by the mobile phone is in text format but when received by nowsms is in binary format. I want that in nowsms also it should be in text format (like in mobile phone). Please revert what is to be done. Thanks
Shailendra Bansal
New member
Username: Embarc

Post Number: 3
Registered: 01-2007
Posted on Wednesday, January 10, 2007 - 06:45 am:   

Going through the different threads I think the key lies in the @@BINARY@@ command to process incoming binary message and convert to text. Can you please explain how to use it?
Malcolm - Now Support
Moderator
Username: Malcolm

Post Number: 82
Registered: 12-2006
Posted on Wednesday, January 10, 2007 - 07:09 pm:   

If a 2-way command is HTTP based, and includes "@@BINARY@@" in the URL template, then NowSMS will execute the associated 2-way command whether the message is in text or binary format.

If "@@BINARY@@" is not in the URL template, then only text format messages will be forwarded to a 2-way command.

This setting doesn't really provide any change in format, it just formats the binary message as a hex string of data when passing it to the 2-way command.

This is described in the following thread:

http://support.nowsms.com/discus/messages/485/624.html

If your problem is that the binary messages are not being routed to a 2-way command, but are being left in the "SMS-IN" directory, then this will resolve your problem.

However, it will not help you interpret the binary content, as it will just be a hex string of data.

You say that a normal mobile phone receives the message as a standard text message? If this is the case, then make sure you have the latest version of NowSMS (2006.10.31 is the current download). I know there were some issues where some "DCS" values were being interpreted as binary when they actually should be interpreted as text, and this could be the issue that you are encountering.

If that is not the issue, then I would be curious to see an SMSDEBUG.LOG file that shows the message being received from the modem, as this might help clarify the message format. (To enable the SMSDEBUG.LOG, use the checkbox on the "Serial #" page of the configuration dialog.)

--
Malcolm
Now Wireless Support
Shailendra Bansal
New member
Username: Embarc

Post Number: 4
Registered: 01-2007
Posted on Thursday, January 11, 2007 - 09:36 am:   

Hi Malcom,

My purpose is to get the incoming binary message from the vehicle unit modem be converted to text. I have downloaded the latest version of nowsms but same status. As pointed out earlier when the same incoming message from the vehicle unit is directed to any mobile phone it gives proper text message. I am sending you the SMSDEBUG.LOG Please go through the same (or otherwise) and let me know how to address the issue.

Thanks and Regards

Shailendra Bansal
application/octet-streamSMSDEBUG log file from Shailendra Bansal
SMSDEBUG-Shailendra.LOG (13.5 k)
Malcolm - Now Support
Moderator
Username: Malcolm

Post Number: 95
Registered: 12-2006
Posted on Thursday, January 11, 2007 - 09:51 pm:   

Hi Shailendra,

Thanks. That helps me understand what you are describing.

The message is being sent in a binary format. The DCS (data coding scheme) value of 4 is a standard value for indicating binary content.

So although the message content appears to be 8-bit ASCII data, NowSMS has to assume that the content is binary as that is what the DCS indicates.

Normal text messages are actually sent using a strange 7-bit binary encoding.

Looking at the message in your log, this is what I see:

Hex string representation of 8 bit data:

3E52505633323936322B323732303233342B303738303231323930303033343331323B49443D3030 303530313C

Conversion of text string using 8 bit data:

>RPV32962+2720234+0780212900034312;ID=000501<

I converted that particular string manually (actually using a simple program), there is no function in NowSMS that will do this if the message encoding indicates that it is binary.

3E = >
52 = R
50 = P
56 = V
33 = 3
32 = 2
etc.

If you want to pass binary messages to a 2-way script in NowSMS, you need to include @@BINARY@@ in the 2-way command template. For example,

http://server:port/script.php?sender=@@SENDER@@&data=@@FULLSMS@@&binary=@@BINARY @@

In this example, if the message content is text, then @@BINARY@@ will be replaced with 0, and @@FULLSMS@@ will contain the text of the message.

If the message content is binary, then @@BINARY@@ will be replaced with 1, and @@FULLSMS@@ will contain the hex string representation of the message data.

--
Malcolm
Now Wireless Support
Shailendra Bansal
New member
Username: Embarc

Post Number: 5
Registered: 01-2007
Posted on Friday, January 12, 2007 - 10:36 am:   

Thanks. Can I have the program which you have used to convert the binary data to text?

Thanks.........Shailendra Bansal
Malcolm - Now Support
Moderator
Username: Malcolm

Post Number: 101
Registered: 12-2006
Posted on Friday, January 12, 2007 - 06:44 pm:   

I actually went to a web site for the conversion.

One thing that I learned from Bryce is that for simple conversions like this (or Base64), you can usually do a quick Google Search to find a tool to do a quick conversion.

I don't have anything in particular bookmarked, I just did a Google search, and it led me to the following link: http://www.paulschou.com/tools/xlate/

I pasted the text into the "HEX" form and pressed the button to convert it.

The actual code would be pretty easy ... I found a Perl sample at the following link:

Perl Script: http://www.planet-source-code.com/URLSEO/vb/scripts/ShowCode!asp/txtCodeId!627/lngWid!6/anyname.htm

In PHP, you would just use the pack function:

$hex_string = '3E52505633323936322B323732303233342B303738303231323930303033343331323B49443D303 0303530313C'
$converted_string = pack('H*', $hex_string);

Or, for other languages, you could just do it brute force, stepping through the hex string 2 characters at a time, converting each 2 hex character block to a single ascii character.

--
Malcolm
Now Wireless Support
Shailendra Bansal
New member
Username: Embarc

Post Number: 6
Registered: 01-2007
Posted on Saturday, January 13, 2007 - 10:02 am:   

Thanks Malcom. The exact string received is 0791198901509141040C911989796755110004701011412351220F3E535041393839373736353531 313C

and in this 3E535041393839373736353531313C is the binary data which you have explained how to convert to text.

Can you please explain the significance of the first part of the data i.e.

0791198901509141040C911989796755110004701011412351220F

and how to decode that. Nowsms is doing but I wanted to know what logic is being used.

Thanks........Shailendra Bansal
Shailendra Bansal
New member
Username: Embarc

Post Number: 7
Registered: 01-2007
Posted on Saturday, January 13, 2007 - 03:50 pm:   

One more query - how does nowsms accept both binary and text message in sms-in file although the commands for both are different i.e. AT+CMGF=0 and AT+CMGF=1 ?