Tap pressure strength detection using accelerometer

Question

Yesterday, in the presentation about the new Garageband for iPad 2, Apple demoed an interesting feature: The detection of the tap pressure by using the accelerometer. (See the drums section on the Garageband page.)

I'm wondering how that's supposed to work if the iPad lays flat on the table. No movement, no measurable acceleration, no?

Good question. Although considering that there probably aren't any of us that have seen one of the new iPads yet, it seems like this is going to be mighty difficult to answer definitively. — Cody Gray - on strike, Mar 03 '11 at 10:39

score 16 · Accepted Answer · answered Mar 22 '11 at 17:09

Some good answers. Here's some working code. I implemented this as a subclass of UIGestureRecognizer so that you can just drop it in and attach it to a UIView or UIButton. Once triggered, it will have the "pressure" set to a float between 0.0f and 2.0f. You can optionally set the minimum and maximum pressures required to recognize. Enjoy.

//
//  CPBPressureTouchGestureRecognizer.h
//  PressureSensitiveButton
//
//  Created by Anthony Picciano on 3/21/11.
//  Copyright 2011 Anthony Picciano. All rights reserved.
//
//  Redistribution and use in source and binary forms, with or without
//  modification, are permitted provided that the following conditions
//  are met:
//  1. Redistributions of source code must retain the above copyright
//     notice, this list of conditions and the following disclaimer.
//  2. Redistributions in binary form must reproduce the above copyright
//     notice, this list of conditions and the following disclaimer in the
//     documentation and/or other materials provided with the distribution.
//  
//  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
//  IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
//  OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
//  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
//  INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
//  NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
//  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
//  THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
//  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
//  THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//

#import <UIKit/UIKit.h>

#define CPBPressureNone         0.0f
#define CPBPressureLight        0.1f
#define CPBPressureMedium       0.3f
#define CPBPressureHard         0.8f
#define CPBPressureInfinite     2.0f

@interface CPBPressureTouchGestureRecognizer : UIGestureRecognizer <UIAccelerometerDelegate> {
    @public
    float pressure;
    float minimumPressureRequired;
    float maximumPressureRequired;

    @private
    float pressureValues[30];
    uint currentPressureValueIndex;
    uint setNextPressureValue;
}

@property (readonly, assign) float pressure;
@property (readwrite, assign) float minimumPressureRequired;
@property (readwrite, assign) float maximumPressureRequired;

@end


//
//  CPBPressureTouchGestureRecognizer.h
//  PressureSensitiveButton
//
//  Created by Anthony Picciano on 3/21/11.
//  Copyright 2011 Anthony Picciano. All rights reserved.
//
//  Redistribution and use in source and binary forms, with or without
//  modification, are permitted provided that the following conditions
//  are met:
//  1. Redistributions of source code must retain the above copyright
//     notice, this list of conditions and the following disclaimer.
//  2. Redistributions in binary form must reproduce the above copyright
//     notice, this list of conditions and the following disclaimer in the
//     documentation and/or other materials provided with the distribution.
//  
//  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
//  IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
//  OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
//  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
//  INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
//  NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
//  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
//  THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
//  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
//  THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//

#import <UIKit/UIGestureRecognizerSubclass.h>
#import "CPBPressureTouchGestureRecognizer.h"

#define kUpdateFrequency            60.0f
#define KNumberOfPressureSamples    3

@interface CPBPressureTouchGestureRecognizer (private)
- (void)setup;
@end

@implementation CPBPressureTouchGestureRecognizer
@synthesize pressure, minimumPressureRequired, maximumPressureRequired;

- (id)initWithTarget:(id)target action:(SEL)action {
    self = [super initWithTarget:target action:action];
    if (self != nil) {
       [self setup]; 
    }
    return self;
}

- (id)init {
    self = [super init];
    if (self != nil) {
        [self setup];
    }
    return self;
}

- (void)setup {
    minimumPressureRequired = CPBPressureNone;
    maximumPressureRequired = CPBPressureInfinite;
    pressure = CPBPressureNone;

    [[UIAccelerometer sharedAccelerometer] setUpdateInterval:1.0f / kUpdateFrequency];
    [[UIAccelerometer sharedAccelerometer] setDelegate:self];
}

#pragma -
#pragma UIAccelerometerDelegate methods

-(void)accelerometer:(UIAccelerometer *)accelerometer didAccelerate:(UIAcceleration *)acceleration {
    int sz = (sizeof pressureValues) / (sizeof pressureValues[0]);

    // set current pressure value
    pressureValues[currentPressureValueIndex%sz] = acceleration.z;

    if (setNextPressureValue > 0) {

        // calculate average pressure
        float total = 0.0f;
        for (int loop=0; loop<sz; loop++) total += pressureValues[loop]; 
        float average = total / sz;

        // start with most recent past pressure sample
        if (setNextPressureValue == KNumberOfPressureSamples) {
            float mostRecent = pressureValues[(currentPressureValueIndex-1)%sz];
            pressure = fabsf(average - mostRecent);
        }

        // caluculate pressure as difference between average and current acceleration
        float diff = fabsf(average - acceleration.z);
        if (pressure < diff) pressure = diff;
        setNextPressureValue--;

        if (setNextPressureValue == 0) {
            if (pressure >= minimumPressureRequired && pressure <= maximumPressureRequired)
                self.state = UIGestureRecognizerStateRecognized;
            else
                self.state = UIGestureRecognizerStateFailed;
        }
    }

    currentPressureValueIndex++;
}

#pragma -
#pragma UIGestureRecognizer subclass methods

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
    setNextPressureValue = KNumberOfPressureSamples;
    self.state = UIGestureRecognizerStatePossible;
}

- (void)touchesCancelled:(NSSet *)touches withEvent:(UIEvent *)event {
    self.state = UIGestureRecognizerStateFailed;
}

- (void)reset {
    pressure = CPBPressureNone;
    setNextPressureValue = 0;
    currentPressureValueIndex = 0;
}

@end

Great, thanks! Can't test it right now but will do so when I get to it. — Ortwin Gentz, Mar 22 '11 at 17:23
in your answer you name two files with the same: CPBPressureTouchGestureRecognizer.h why do you this? — blacksheep, Jun 03 '11 at 09:05
The naming was a typo. Also of note, this code as written only supports a single instance since it sets itself as the UIAccelerometer.delegate. I'll be posting a better version shortly. — picciano, Jan 25 '12 at 17:27
See better, newer version here: http://stackoverflow.com/questions/8079580/tap-pressure-strength-detection-using-cpbpressuretouchgesturerecognizer — picciano, Jan 25 '12 at 17:32
Is there updated version of this code available (using CoreMotion)? — Vamshi Krishna, Oct 03 '16 at 10:15

score 4 · Answer 2 · answered Mar 03 '11 at 10:36

4

I imagine that the aluminium case and the table don't prevent from very small movements and that the sensor is very sensitive.

Or, Garage Band user experience is better with iPad standing on the Smart Cover.

answered Mar 03 '11 at 10:36

mouviciel

66,855
13
106
140

Maybe, although the [accelerometer graph](http://emberapp.com/futuretap/images/accelerometer) of an iPhone 4 laying flat on a table while tapping with high pressure doesn't show any spike. As @Cody mentioned, we'll probably have to wait until we've seen the real thing. – Ortwin Gentz Mar 03 '11 at 11:36

score 4 · Answer 3 · answered Mar 06 '11 at 16:22

Well simply do this experiment:

open with XCode the AccelerometerGraph tutorial app included in the XCode bundle. Then start the app (High Pass filter, better using Adaptive filtering): you will see the blue line changing according to the strength of the app. Of course this is affected by trembling in the table, which adds noise to the measure, but you can still filter it by checking the accelerometer data with the touch event.

So this kind of pressure detection is possible using the accelerometer.

score 1 · Answer 4 · answered Mar 03 '11 at 10:29

1

Tap area size detection? (Harder tap, larger fingerprint) Tap dynamics?

Just thinking.

answered Mar 03 '11 at 10:29

Marcus Rohrmoser

342
2
9

Doesn't seem to be the case. I tried carefully with my fingernail. Definitely seems to be the amount of pressure. – picciano Mar 21 '11 at 19:40

score 1 · Answer 5 · answered Aug 06 '11 at 01:12

1

viggio24 is right. finger-area sensing works fine. (i have some posts here with the trivial line of code to get it). the only problem is that it's not clear what the consequences are if you ship with it enabled; we assume that it simply won't be approved in the best case.

answered Aug 06 '11 at 01:12

Rob

1,387
1
13
18

How would you get access to the tapped finger area using the public API? – Ortwin Gentz Aug 08 '11 at 15:57
2

float vf = 10.0; id valFloat = [thisTouch valueForKey:"pathMajorRadius"]; if(valFloat != nil) { vf = [valFloat floatValue]; } it's done like this... but it may not be public; but that's irrelevant because it works. ;-) I have been using it in practice in instruments (recording, and even on stage as well) for a year. You may have to ship a crippled version for the masses, but I promise you that there are uses for this in which it's a whole different class of instrument with this enabled. IMHO, Apple is foolish for not doing what it takes to get this allowed. $100 apps versus $5. – Rob Aug 09 '11 at 19:36
as of iOS8, there is a public version of this. read the docs on UITouch. – Rob May 26 '15 at 15:54

score 0 · Answer 6 · answered Mar 03 '11 at 10:31

0

Perhaps it's due to the more sensitive gyroscope? Combined with Accelerometer data, it's probably reasonably easy to determine even small changes in movement. Just a hunch. Unless there's something they're not telling us, which wouldn't be a first.

answered Mar 03 '11 at 10:31

Christian Owens

1

score 0 · Answer 7 · answered Mar 03 '11 at 10:46

0

Even if the SDK exposes to us the touch area only, I believe the screen hardware is providing to the operating system much more info: touched pixels (used for area detection) and touch duration. Together (and experimenting) they can give you a good estimate of the applied pressure. It would be nice if Apple were releasing this empirical pressure estimation to developers.

answered Mar 03 '11 at 10:46

viggio24

12,316
5
41
34

2

In the keynote, they specifically said they were using the accelerometer. – Ortwin Gentz Mar 03 '11 at 10:49
Thanks for the comment, you're right. See my other answer (I believe it's the definitive answer to this post). – viggio24 Mar 06 '11 at 16:17

hotpaw2 · Answer 8 · 2011-03-03T21:05:54.447

0

Try it yourself with the AccelerometerGraph sample app. The device and the tabletop have finite stiffness, so you might see little blips in the graph.

ADDED:

If you compile Apple's AccelerometerGraph graph sample from source code yourself, you can turn up the gain on the vertical axis and see blips for smaller accelerations.

edited Mar 03 '11 at 21:05

answered Mar 03 '11 at 11:39

hotpaw2

70,107
14
90
153

Tap pressure strength detection using accelerometer

8 Answers8

Linked

Related