Inconsistencies when measuring "strength" of finger taps

API, R&D and Technology
1

I am experimenting with using the Sensel Morph as an input device for finger drumming, i.e. triggering drum samples by tapping with the fingers on the sensor surface. The idea here is to use the x-y position to determine the drum and articulation, and use the “strength” of the tap for the velocity.

I am using the C API to look for contacts and trigger MIDI notes accordingly.

Things that seem to work well for this application:

  1. The latency seems to be acceptably low, at least when using the low resolution / high scanning frequency setting. I set it to SCAN_DETAIL_LOW and set the max frame rate to 1000Hz. (Note: Even though I can successfully set the value to 100, I assume I am actually only getting 500Hz, since the specs says the actual maximum at low scan detail is 500Hz. Haven’t tested how many frames per second it actually delivers).
  2. The sensitivity range is wide enough to detect even light taps, which is great.
  3. Detection of multiple rapid and potentially simultaneous contacts works well (e.g. for drum rolls).

The one thing that does not seem to work well is to use the force readings to derive the “strength” of the tap. The issue I am seeing is that the readings are very inconsistent. For taps of very similar perceived strength, I see the value of the SenselContact.total_force of the first frame detecting the contact vary by as much as a factor 100. I did a couple of tests to try to understand why this happens and if there is any workaround:

  1. To make sure this is not due to my test taps being too inconsistent, I used a badminton shuttlecock and dropped it repeatedly from the same height. I still got force readings vary by as much as factor 10.

  2. To check if the inconsistency is an artifact of the algorithm that determines contacts from the force image frames, I also compared with the total force accumulated from the force image frame (making sure there is only ever one contact at a time), and the inconsistence is present there as well.

Currently my best guess as for a cause is that this could be a timing issue caused by the force readings corresponding to essentially instants in time. Depending on at what stage of the tap the force sample gets taken, one might end up with very different values. For example, if a sample is taken at a moment where the finger has just barely started touching the sensor and has not yet reached its lowest point, the force reading might be low. If the sample happens to get taken at the time where the tap exhibits the highest force, then one may end up with a very high value.

Note, that the above theory also matches observations that can be made using the visualizer in the Sensel App. When tapping the sensor such that the duration of the contact is very short, one can observer that sometimes the visualizer shows a large peak and sometimes it shows nothing.

Can anyone confirm whether or not my theory for the cause is correct?
If so, then it might be that this type of sensor is simply not well suited for measuring the strength of very brief contacts.

Please let me know if you have any suggestions as for how the Morph could be used to determine the strength of finger taps.

2

Hello, thank you for reaching out. Yes, your theory is correct. A tap can be measured at different times in a strike so the total_force would be different at that frame. We are working on a few ways to improve velocity sensing and include this data into the API as well. As for the visualizer, it is only running at 30 FPS so it cannot show all of the data when running at 500+ Hz. Until we provide more velocity data in the API, I would recommend looking at the difference between two frames: either between the first two frames of the contact or use the force frames from the first frame of the contact and the previous frame to calculate an approximate velocity.

3

Thanks, Alex, for the quick response and the very useful insights and suggestions.

I followed your advice and did the following (assuming for now that there is never more than one contact/tap at a time). When the contact starts, I keep track of the total frame force starting at the frame right before the contact until the first frame where the force has decreased. From the frame where the force has decreased, I use the last 3 frames to calculate the peak of a fitted parabola as an estimate of the tap strength.

The results are more consistent than what I had before, but there still seems to be some randomness. In some cases, I would estimate maybe 1 our of 20 times, I end up with an estimate that is much higher than the others, caused by the middle of the 3 frames having a much higher value. A possible explanation could be that the tap is too fast/short/transient to be reliably captured with the respective sample rate. I would end up with a high value only in cases where the peak force of my tap happens to coincide with a sample position.

By printing a message every 1000 frames received, I found that I get roughly 2000 frames each 3 seconds, which corresponds to ~600Hz.

Not sure if the above explanation and theory makes sense. I find it surprising that finger tapping could be too fast to fully capture at 600Hz. But if that really is the case, would you be able to internally run at a higher sampling rate (at least in short bursts) to capture such transient events more accurately? I’d be curious to learn what strategies you are pursuing to improve velocity sensing, but I of course understand if you are unable to share that information.

Best Regards,

Christian

4

Thanks for sending this followup message. Yes, this does replicate some of the behavior we have seen in our API mode. In MIDI/MPE mode with overlays, we are already doing some additional processing so you will never miss a hit and offer low latency (2 ms with drum overlay, 5-6ms with other overlays). We will continue to offer more information going forward on how to get access to more information in the API mode. Thank you for your patience and support!