Calculate the distance between the centre point of the palette and the centre point of the picker circle (How to find the distance between two CG points?)
Add on the radius of the picker circle and if that exceeds the radius of the larger circle then you want to stop the dragging.
UPDATE:

So if the Distance + R2 is >= Radius 1 you've reached the edge of the circle and need to stop them dragging
UPDATE2
Based on the sample project you uploaded here is the correct code...
- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event {
for (UITouch *touch in touches){
CGPoint newPoint = [touch locationInView:self];
newPoint.x -= startPoint.x;
newPoint.y -= startPoint.y;
CGRect frm = [picker frame];
frm.origin = newPoint;
CGFloat xDist = abs((newPoint.x + 15) - (self.center.x - self.frame.origin.x));
CGFloat yDist = abs((newPoint.y + 15) - (self.center.y - self.frame.origin.y));
CGFloat distance = sqrt((xDist * xDist) + (yDist * yDist));
if ((distance + 15) >= (self.frame.size.width /2)) {
// EDGE REACHED SO DON'T UPDATE
} else {
[picker setFrame:frm];
}
}
}
You only want to update the frame if its still in bounds and your distance calculations weren't taking into account the offset of the containing view
HTH