VGuitar Forums

MIDI Controllers / Interface / Tone Modules => MIDI Controllers / Interface / Adapters => Topic started by: Elantric on April 20, 2013, 04:38:10 PM

Title: IOS MIDI over Bluetooth
Post by: Elantric on April 20, 2013, 04:38:10 PM
https://web.archive.org/web/20130105034619/http://www.jaycoggin.com/home/ios-midi-over-bluetooth (https://web.archive.org/web/20130105034619/http://www.jaycoggin.com/home/ios-midi-over-bluetooth)

iOS MIDI Over Bluetooth
It's finally under development!  The ability to send and receive MIDI messages to a small wireless Bluetooth box with standard MIDI In/Out ports using an iOS device.  I can't put a timeframe on the process because I have so much on my plate, but I'll show below what I've got so far and where I plan on going with it.

Overview

Below is a sort of overall diagram for the system.  The iDevice sends and receives MIDI messages with a small external piece of hardware via Bluetooth Low Energy (BLE).  This new type of Bluetooth is responsible for making this whole project possible, and is only supported on iDevices beginning with the iPhone 4S and newer.  This includes the iPhone 5, iPad 3 and 4, newest iPod Touch, and iPad Mini.  The small piece of hardware contains a MIDI In and Out port and communicates these messages with the iDevice.  MIDI Out messages can be used to drive, for example, a DAW or hardware synth, and the MIDI In port can receive and forward controller messages to the iDevice.

(https://www.vguitarforums.com/smf/proxy.php?request=http%3A%2F%2F1.bp.blogspot.com%2F-JrgzuzEpVi4%2FUJDSTYWdlxI%2FAAAAAAAFtuw%2FPEzFK8QQuFA%2Fs1600%2FFlowchart-OG-1024x702.png&hash=7b8be6604ace0b04cb8637cc5a16ce54e8027072)

Hardware

Here is a picture of my prototype piece of hardware that would send/receive the MIDI messages with the iDevice and the external MIDI device.  The two boards pictured are identical, only one is needed.  The board has a MIDI In and Out port and an RS-232 serial port that is just used to load new firmware to the Bluetooth chip.  Eventually, this would be removed and a USB Mini port put in its place for updating the firmware (if necessary).  Currently, the MIni USB port is just used to power the board.

(https://www.vguitarforums.com/smf/proxy.php?request=http%3A%2F%2F4.bp.blogspot.com%2F-Ixc_4LgdDzk%2FUJDTWYioanI%2FAAAAAAAFtu8%2F5kKez2nQolc%2Fs1600%2FIMG_0402-1024x764.jpg&hash=942fbe993464b7699035825d973daee354556ab2)
Functionality

In addition to the hardware, I have written the firmware to be run on the Bluetooth microprocessor and the appropriate MIDI to BLE iOS classes for sending and receiving messages.  By implementing these classes into my work-in-progress ROTOSynth app (see tab, a fully functioning controller and synthesizer iPad app), I am able to both control an external synth using my app's controller by sending MIDI messages to my hardware (and thus, out the MIDI Out port), or play my app's synthesizer using an external MIDI controller (connected to the MIDI In on my hardware).  Although already outdated, here's a video of when I first got one side of the communication going (iPad controller to MIDI Out).


Latency

Now, the most important question everyone will ask is, "Ok, cool, but how does it stack up against the existing ability to send/receive MIDI messages over Wi-fi on iDevices?".  In short, it beats it – by a lot.

Below are two graphs which show the results of my latency testing of various performance scenarios.  To test, I used two microphones connected to my PC through a Presonus Firebox recording simultaneously in my DAW.  One mic was placed right next to the appropriate controller (a key on the MIDI controller or the screen on my iPad, depending on the test), and the other was placed at the sound source (either the studio monitor or iPad speaker, depending).  For each test, I would begin recording on the PC, then initiate 10 Note On events on the appropriate controller.  I struck the controller assertively so as to capture the touch event on the controller microphone.  Then, by manual analysis in the DAW, I was able to see the latency between the controller touch event and the corresponding sound.  Below is the average latency over 10 trials for each scenario.



(https://www.vguitarforums.com/smf/proxy.php?request=http%3A%2F%2Fwww.jaycoggin.com%2Fhome%2Fwp-content%2Fuploads%2F2012%2F10%2FMIDIBLE_Graph_Latency-1024x894.png&hash=ee59ce523bda68a874d6e0bae59f0377efe4591a)

Each schencario is explained a little more in depth here:

Axiom to Logic w/ MIDI Cable – Essentially the control.  A hardwired MIDI controller (M-Audio Axiom 49) into my USB Focusrite interface feeding into Logic.
iPad to Logic w/ BLE – ROTOSynth acting as a BLE controller.  External BLE device connected to MIDI In on Focusrite interface and feeding Logic
Axiom to iPad w/ BLE – ROTOSynth acting as the synthesizer.  Axiom controller connected to MIDI In on BLE device and controlling the Core Audio synth in my app.
iPad All Local – ROTOSynth acting as both controller and synthesizer.  Tested to get idea of baseline for iDevice synthesis app latency (using Audio Units)
SoundPrism to Logic w/ Ad-Hoc – Very popular iPad controller app that implements the Core MIDI framework.  Used for baseline of Core MIDI latency over 802.1 Ad-hoc network
SoundPrism to Logic w/ Wi-Fi – Same as above but using an 802.1n local Wi-Fi network rather than ad-hoc network.
As can be seen, the latency with BLE is about 30 ms better than with the best case scenario using Core MIDI's implementation over Wi-fi.  Even more, look at the graph below, which shows the Standard Deviation (SD) of the 10 latency tests for each scenario.  For those not familiar, the standard deviation gives a measure of how much the data varies around the average value.  In this context, it tells us how much the latency varies from one note to the next.  A high SD means the latency is all over the place, a low SD means the latency is very consistent.


(https://www.vguitarforums.com/smf/proxy.php?request=http%3A%2F%2Fwww.jaycoggin.com%2Fhome%2Fwp-content%2Fuploads%2F2012%2F10%2FMIDIBLE_Graph_SD.png&hash=385e50bddd709a850c167bb9be3883801f657a7d)

The SD using BLE for MIDI is remarkably lower than with MIDI over Wi-fi.  This means that the latency is more "learnable" so that musicians can adapt and not be left guessing at when the note they press will actually be played.

Versatility

Now, the fastest way for users to get up and running with MIDI over BLE is for me to release the hardware along with a graphically basic app that supports Virtual MIDI.  By doing so, my app could receive and transmit MIDI messages between other iOS apps and the external piece of MIDI hardware connected.  However, that's the most basic application.  Multiple BLE devices can be connected at once – and each with their own input/output data stream, so ideally, you could control, or be controlled, by several external MIDI devices.  Even further, a MIDI to DMX (lighting) box would be totally possible, so iOS apps could implement full control over live venue lighting!  Beyond the basic implementation to get MIDI over BLE performing the functions of MIDI over Wi-fi, there's countless more possibilites!

Interested?

If this sounds like something you'd like to use, either as an end-user or developer wanting to make cool, multi-MIDI device apps, comment below to let me know!  With enough interest, I would certainly pursue this with a kickstarter project for the hardware and a simple iOS app for interfacing with Virtual MIDI apps.  If developers want to use multiple devices to start implementing cool multi-device control, I'll consider releasing an SDK.  You guys determine it, so just let me know!

Progress Update:

http://www.jaycoggin.com/home/ios-bluetooth-midi-update/ (http://www.jaycoggin.com/home/ios-bluetooth-midi-update/)


Some important clarification here on Bluetooth MIDI
http://www.iosmusician.com/category/bluetooth (http://www.iosmusician.com/category/bluetooth)

http://m.matrixsynth.com/2012/10/wireless-midi-bluetooth-box-finally.html (http://m.matrixsynth.com/2012/10/wireless-midi-bluetooth-box-finally.html)
Title: Re: IOS MIDI over Bluetooth
Post by: aliensporebomb on April 21, 2013, 11:13:02 AM
Really intrigued.  Very promising.