I’m wondering what the current capabilities are of the Morph for it’s non-USB-to-host connections.
I’ve read that there are multiple levels to receiving touch data from the Morph. One level can be Overlay interaction where overlay “keys” & force. The next can communicate the “Contacts” that describe arbitrary points where the force is being applied. And the lowest just shows the force array “image” that can be analyzed by the computer instead of the Morph.
My question is, given the limited bandwidth of the Bluetooth connection and that of the USB-Serial Dev Cable, which of the aforementioned (and not mentioned) data can be accessed by each type of non-USB connections?
Hello, thank you for contacting us. Currently our Bluetooth does not support a serial connection. As a result, you can only get contacts and force frames using USB and our open source API https://github.com/sensel/sensel-api.
Separately, our Overlays allow you to send standard mouse, keyboard, gamepad, MIDI and several other events. Each area on the Overlay can be customized to send any one of these standard events. You can create your own maps using our SenselApp software that can send these maps to the Morph. Bluetooth currently supports MIDI, mouse, and keyboard events.
Another option for Bluetooth is to use MPE (MIDI Polyphonic Expression) to get multitouch information over Bluetooth. If you set up the No Overlay mode as a MPE XYZ Pad with absolute bounds, each contact would be on on a different channel. On each contact, pitchbend, cc74 and channel aftertouch (or whatever you want to map it to) can be used to get XYZ information about each contact. I would also recommend using 14-bit CC if you need more accurate data.
With regards to your other questions:
We currently do not have a decompression library for the developer’s cable. There is probably some way to use developer’s cable->UART Bluetooth module->Sensel Open Source API, but I have not done it before myself.
Yes, it is locked in to 115200. However, with the amount of data being sent, communication should not be the bottleneck.
I do not believe this is open-sourced. I will check internally and if it is, I will follow up.
Oh excellent! I will look into that and see if it’ll be sufficient for what I need.
Should I interpret the missing “decompression library” to mean Arduino-side decoding is not implemented, but the raw compressed data should still be available accessible through the Dev-Cable’s serial interface? If that’s the case, is your decompression source also opened-sourced for the USB interface library? If so I would think it shouldn’t be exceptionally difficult for me to port it over.
Oh cool! I hadn’t run the numbers to see what realtime Force Array scanning would consume bandwidth-wise, and thereby had assumed it wouldn’t be fast enough.
1 . Upon further reading I see that the decompression algorithm is not open source. Still, is the compressed data accessible through the serial interface?
4 . Which processor architecture(s)/OSes are the library binaries compatible with? My platform(s) may include Android and/or Ubuntu.
1.Yes, the decompression library is not open source but the data is still accessible through the serial interface. As long as you pass the same data to the library on Mac, Windows or Linux, the decompression library should work
4.We currently have compiled binaries for Mac, Windows, Linux (64bit, 32bit and arm). You can also build the libraries without the decompression library (no force frames, but you can get contacts). This should work on Android (given proper serial communication permissions).
2.So just to confirm, uninterrupted 115200 baud is fast enough to carry the force frames at the max refresh rate?
20k “pixels” * 15 bits * 125 Hz = 37.5 mbps. I would assume this is where the compression comes into play to squeeze it beneath the theoretical max 12 mbps of the USB connection
4.Is the ARM binary compiled for 32-bit or 64-bit processors?
So I set the “No Overlay” to “MPE XYZ Pad”, Set X → CC1, Y → CC2, Pressure → CC3, Threshold → 0, and enabled Absolute X & Y, 14-bit CC, and Recenter CC. Using a “Processing” sketch with The MidiBus library and some basic MIDI byte-dumping print statements, with the Morph plugged in via USB I used multiple fingers on the Morph and could see the contact ID, X & Y location, and contact pressure being continuously streamed. So the Morph can definitely format the desired multitouch data into a MIDI/MPE format.
However when using the BLE interface instead of USB I see 3 MIDI messages each time when I remove a “contact”/finger from pressing, and no other data besides that. And that’s with an unchanged No Overlay configuration. Additionally, the wifi for the computer was turned off, and the Morph was right next to the computer - so I wouldn’t expect interference to be causing any problems. Especially since there were always the 3 packets that were received.
And in case it’s of any help, only the last byte/parameter of the third byte had any variation (for single contacts). Multiple contacts were detected and given unique IDs, however those too would only send their three MIDI messages when each contact was removed.
Is this the expected behavior of the Morph in an MPE mode over Bluetooth?
If there are any additional tests/diagnostics that I could do to help identify the problem, assuming there is one, please let me know. It would be awesome to be able to use the Morph cordlessly as-is without having to build a data-path hardware workaround.
Hello, thank you for contacting us. You should be seeing the following data over Bluetooth. I did notice that 14-bit CC did drop a few messages so you may want to disable that checkbox for now. If you continue to experience errors, feel free to contact us at support@sensel.com. Thank you.
On OSX 10.10 I used MIDI Monitor to observe the incoming MIDI data via Bluetooth. When I have the 14-bit CC disabled in the No Overlay “MPE XYZ Pad” then the expected data is sent to the computer - a continuous flow. But if 14-bit CC is enabled then only when a contact is released are there three lone MIDI packets that are received by the computer. So it does appear that the 14-bit CC setting is what’s triggering the unusual behavior. This issue only occurs when using Bluetooth & 14-bit CC is enabled. Using USB with 14-bit CC enabled works without issue.
If the BTLE MCU were simply being swamped with too much data then I’d expect the packets that were transmitted to the computer to be random and not perfectly patterned/repeatable as it is. Perhaps the main processor firmware that has trouble forming the MIDI data on Press and Hold events with the higher precision data, but works correctly with the Release events Or perhaps the wireless MIDI data formatting is done on the Bluetooth MCU, and it’s taking issue with the change of precision/packet formats.
Aaaaaanyway I can only speculate. Hopefully you’ll be able to identify the change in the code path that the 14-bit CC + Bluetooth mode triggers.
What additional information can I collect or things can I test that’ll help reproduce the issue on your side and help identify the reason behind the issue?
Are there any diagnostics I could run or setting-combinations I could try that would help narrow down the issue?
Or perhaps the wireless MIDI data formatting is done on the Bluetooth MCU, and it’s taking issue with the change of precision/packet formats.
