Some time ago i soved a similar problem in such way. Maybe it will be helpful for you
$str = 'sqrt((2 * (2+2)) + sin(pi/(6+7)))';
$from = []; // parentheses content
$to = []; // patterns for replace #<number>
$brackets = [['(', ')'], ['{', '}'], ['[', ']']]; // new parentheses for every level
$level = 0;
$count = 1; // count or replace made
while($count) {
$str = preg_replace_callback('(\(([^()]+)\))',
function ($m) use (&$to, &$from, $brackets, $level) {
array_unshift($to, $brackets[$level][0] . $m[1] . $brackets[$level][1]);
$i = '#' . (count($to)-1); // pattern for future replace.
// here it '#1', '#2'.
// Make it so they will be unique
array_unshift($from, $i);
return $i; }, $str, -1, $count);
$level++;
}
echo str_replace($from, $to, $str); // return content back
// sqrt[{2 * (2+2)} + sin{pi/(6+7)}]
I forgot all details, but it, seems, works