Boss MS-3 - MIDI over USB Sysex Reverse Engineering

Started by MrHaroldA, August 29, 2017, 12:13:37 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

MrHaroldA

Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

vtgearhead

Quote from:  MrHaroldA on September 24, 2017, 03:03:35 AM

uint8_t checksum(uint8_t *parameter, uint8_t *data, uint8_t dataSize) {
    uint8_t sum, i;

    for (i = 0; i <= 3; i++) {
        sum = (sum + parameter[i]) & 0x7F;
                   ------  Bzzzt!
    }
    for (i = 0; i < dataSize; i++) {
        sum = (sum + data[i]) & 0x7F;
    }

    return (128 - sum) & 0x7F;
}



Um, unless I'm missing something 'sum' is uninitialized at the first point it's used.  See buzzer above.


MrHaroldA

Quote from:  snhirsch
You are correct - I just fixed my posting above. 

To clarify:


  • These commands DO NOT require the usual sysex prefix / suffix and checksum
  • The command to enter edit = 7F, 00, 00, 01, 01
  • The command to exit edit = 7F, 00, 00, 01, 00


That still doesn't work...

Sniffing the Editor startup revails these messages that are sent first:


SYSX: F0 7E 00 06 02 41 3B 03 00 00 00 00 00 00 F7
SYSX: F0 7E 00 06 02 41 3B 03 00 00 00 00 00 00 F7
SYSX: F0 41 00 00 00 00 3B 12 7F 00 00 00 00 01 F7


The last one mildly resembles the editor_mode code posted above.

After that, a repetitive number of 30 byte SysEx messages appear.

Edit: full log in the attachment!
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

sixeight

QuoteF0 41 00 00 00 00 3B 12 7F 00 00 00 00 01 F7

41 00 00 00 00 3B is the identifyer for Boss MS3
12 means write data (11 is data request)
7F 00 00 00 is the address
00 is the data
01 the checksum (80 - sum of data and address: 7F + 00 +00 +00 +00 = 7F, 80 - 7F = 01

Try sending F0 41 00 00 00 00 3B 12 7F 00 00 00 01 00 F7

vit3k

For Katana I'm sending whole sysex message with checksum to enable editor mode as for any other command. Here is my method for calculating checksum working on Arduino. You can use it if you want.



byte Katana::calculateChecksum(byte* vals, byte valsSize)
{
    byte acc = 0;
    for(byte i = 0; i < valsSize; i++)
    {
        acc = (acc + vals[i]) & 0x7F;
    }

    return (128 - acc) & 0x7F;
}


Vals has to point to the beginning of address with correct size (4 for address + some for data). I don't know if the same works on MS-3.

vtgearhead

Quote from:  sixeight on September 24, 2017, 08:24:59 AM
41 00 00 00 00 3B is the identifyer for Boss MS3
12 means write data (11 is data request)
7F 00 00 00 is the address
00 is the data
01 the checksum (80 - sum of data and address: 7F + 00 +00 +00 +00 = 7F, 80 - 7F = 01

Try sending F0 41 00 00 00 00 3B 12 7F 00 00 00 01 00 F7

And you are of course correct - I'll delete my earlier, misleading posts.  Sorry about that!

MrHaroldA

Quote from:  sixeight on September 24, 2017, 08:24:59 AM
Try sending F0 41 00 00 00 00 3B 12 7F 00 00 00 01 00 F7

Is that 00 in the right position?

Sending "F0, 41, 0, 0, 0, 0, 3B, 12, 7F, 0, 0, 0, 0, 1, F7" didn't do the trick, neither did "F0, 41, 0, 0, 0, 0, 3B, 12, 7F, 0, 0, 0, 1, 0, F7"

I'm off to bed now; the hunt continues tomorrow!
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

vtgearhead

#32
Now that I've re-read the sources, this is what I used to put a Katana into edit mode:

0xF0, 0x41, 0x00, 0x00, 0x00, 0x00, 0x33, 0x12, 0x7f, 0x00, 0x00, 0x01, 0x01, 0x7F, 0xF7

And this should cancel edit mode:

0xF0, 0x41, 0x00, 0x00, 0x00, 0x00, 0x33, 0x12, 0x7f, 0x00, 0x00, 0x01, 0x00, 0x00, 0xF7

Given how many times I stuck my foot in my mouth during my participation in this thread I'd appreciate someone else reviewing this post for correctness.

UPDATE:  All I know is that the strings above are (a) what is calculated by the checksum code in my program and (b) work properly.  I was looking at a post that showed different checksum bytes.


vtgearhead

Quote from:  sixeight on September 24, 2017, 08:24:59 AM
41 00 00 00 00 3B is the identifyer for Boss MS3
12 means write data (11 is data request)
7F 00 00 00 is the address
00 is the data
01 the checksum (80 - sum of data and address: 7F + 00 +00 +00 +00 = 7F, 80 - 7F = 01

Try sending F0 41 00 00 00 00 3B 12 7F 00 00 00 01 00 F7

Six-Eight:  Are you sure about that algorithm?  I've always used this:

# Example: Take the address (4 bytes) and data (1 byte) in this case
vals = [0x60, 0x00, 0x12, 0x14, 0x01]

# Sum with 7-bit wraparound
accum = 0
for val in vals:
    accum = (accum + val) & 0x7F

# Checksum is lower 7 bits of the difference w/ 128
cksum = (128 - accum) & 0x7F

Your post is not doing a difference with 128.

sixeight

QuoteIs that 00 in the right position?

Yes. It is the checksum (7F + 00 + 00 + 00 + 01 = 0x80; ox80 - 0x80 = 00)

Maybe these will work:
F0 41 00 00 00 00 3B 12 7F 00 00 01 01 7F F7
F0 41 00 00 00 00 3B 12 7F 00 00 02 01 7E F7

MrHaroldA

None of these options work, so I replicated my Arduino program in a Linux bash script:


#!/bin/bash
MIDI='hw:2,0,0'

echo "Setting editor mode" # first three messages from the editor start.
amidi -p ${MIDI} -S 'F0 7E 00 06 02 41 3B 03 00 00 00 00 00 00 F7'
amidi -p ${MIDI} -S 'F0 7E 00 06 02 41 3B 03 00 00 00 00 00 00 F7'
amidi -p ${MIDI} -S 'F0 41 00 00 00 00 3B 12 7F 00 00 00 00 01 F7' # This already triggers BULK DATA

while true; do
  echo "Setting FX1 ON"
  amidi -p ${MIDI} -S 'F0 41 0 0 0 0 3B 12 60 0 0 30 1 6F F7'
  sleep 1

  echo "Setting FX1 OFF"
  amidi -p ${MIDI} -S 'F0 41 0 0 0 0 3B 12 60 0 0 30 0 70 F7'
  sleep 1
done


This doesn't need compilation, upload, verify, reboot, serial terminal, etc, so it's way faster to try stuff.

Anyway; this doesn't work either. FX1 toggles nicely, but with the "BULK DATA RECEIVING..." overlay in the display even by setting the editor mode exactly like the editor does.

I'm off again, have to sell my condo today ;) See you guys later, and thank you for all your suggestions!!!
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

MrHaroldA

I can't find any other commands the Editor/Librarian sends than those three posted above; and even mimicking the timing does not prevent the display from displaying "BULK DATA RECEIVING..." upon the third SysEx command.

This is becoming very annoying.
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

vtgearhead

There must be other traffic.  How are you intercepting communication?   At Codesmart's suggestion I've used a low-level MIDI analyzer package that shows absolutely everything in both directions.  Since you'll be seeing raw USB it does require some interpretation to get rid of the data length bytes.  When diving into the Katana last year, I captured bulk data and processed out the length bytes with a Perl script.

gumtown

#38
This one definitely works for me

F0 41 00 00 00 00 3B 12 7F 00 00 01 01 7F F7

Puts the MS-3 into "verbose mode" and stops the "Bulk Data" messages, and it also sends and knob turn/parameter changes back out the Midi/USB.

I am using Bome's SendSX to read and send midi, and a partially constructed MS-3_FxFloorBoard editor.

The MS-3 internal structure is definitely not based on the GT-100, like the GT-1 and Katana are.
Free "GR-55 FloorBoard" editor software from https://sourceforge.net/projects/grfloorboard/

gumbo

...I'm convinced that all you guys are just really TALKING DIRTY to each other in some kind of encrypted code....   >:(
Read slower!!!   ....I'm typing as fast as I can...

MrHaroldA

Quote from:  snhirsch on September 26, 2017, 02:49:51 PMHow are you intercepting communication?   At Codesmart's suggestion I've used a low-level MIDI analyzer package that shows absolutely everything in both directions.

I'm using MIDI-OX right now, which may be too high level; I don't really know these tools. Any suggestions? Linux or Windows only.
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

gumtown

Quote from:  gumbo on September 26, 2017, 11:30:04 PM
...I'm convinced that all you guys are just really TALKING DIRTY to each other in some kind of encrypted code....   >:(
Figure this out  :-*
59 65 61 68 20 42 61 62 79 20 21 20 54 61 6C 6B
20 64 69 72 74 79 20 74 6F 20 6D 65 20 21 21
Free "GR-55 FloorBoard" editor software from https://sourceforge.net/projects/grfloorboard/

MrHaroldA

Hint: 546865726520617265206f6e6c696e6520636f6e7665727465727320617661696c61626c6520746f20636f6e766572742068657820746f2063686172732e
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

gumtown

Free "GR-55 FloorBoard" editor software from https://sourceforge.net/projects/grfloorboard/

MrHaroldA

Quote from:  gumtown on September 27, 2017, 01:04:34 AM
It might seem odd, but my go to hex/code editor tool is PSPad.

I'm using Atom IDE, but that can't sniff Midi on a low level ...
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

sixeight

Quote from:  gumtown on September 27, 2017, 12:51:00 AM
Figure this out  :-*
59 65 61 68 20 42 61 62 79 20 21 20 54 61 6C 6B
20 64 69 72 74 79 20 74 6F 20 6D 65 20 21 21

Don't let Elantric read that one. He will ban you from this forum. ;D

MrHaroldA

Quote from:  gumtown on September 26, 2017, 09:20:11 PM
This one definitely works for me

F0 41 00 00 00 00 3B 12 7F 00 00 01 01 7F F7

I sure hope you edited that HEX code, or else I can't copy/paste like sh*t. :D

After decoding the sniffed USB Analyzer data, I reconstructed "F0 41 00 00 00 00 3B 12 7F 00 00 01 01 7F F7" as one of the first things the editor sends. This is different than what MIDI-OX logs!

(but it's exactly what you posted)

And ... IT WORKS!!!  ;D
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

gumtown

Which is why ii don't use midiOX that much,
I still have 8 ports of MidiYoke installed, but prefer to use
Bomes SendSX
https://www.bome.com/products/sendsx
Free "GR-55 FloorBoard" editor software from https://sourceforge.net/projects/grfloorboard/

MrHaroldA

Ok, setting data works fine and reliable. Now I have to figure out how to receive data from the MS-3 ... I'll probably first sniff what I can expect, and then look into the Arduino Midi Host Shield software on how to actually receive the variable length SysEx data.

I also have to start on the hardware prototype, it will make it all way more "touchable".
Boss MS-3 and Katana library for Arduino: https://github.com/MrHaroldA/MS3

gumtown

#49
Data retrieval is quite straight forward.

F0 41 00 00 00 00 3B 11 60 00 XX XX 00 00 ZZ ZZ CS F7"
where
F0 41 00 00 00 3B = usual boring sysx header stuff
11 = data request mode
60 00 XX XX = start address of temp data block
00 00 ZZ ZZ = data size required from start address
CS = checksum
F7 = end of message

You might find some useful Arduino code here
https://github.com/gumtown/V-tone

A 12 knob box with 3 lcd displays



I also started on a MKii version using rotary encoders, but couldn't get the Arduino to scan 12 encorders



Free "GR-55 FloorBoard" editor software from https://sourceforge.net/projects/grfloorboard/