For your workload and indexes, you should try both queries' execution plan and runtime. In either case you would benefit from having an index on name.
I believe that both queries are going to end up with similar plans. Let's check that out.
Create the tables
create table tablea (id int primary key, nm as varchar(50));
create index idx_tablea_nm on tablea(nm);
create table tableb(a_id int, anotherfield varchar(100),
key idx_tableb_id(a_id),
constraint fk_tableb_tablea_id foreign key (a_id) references tablea (id));
Let's do an EXPLAIN
on the first one:
explain select tableb.* from tablea inner join tableb on tablea.id = tableb.a_id where tablea.nm = 'Alice';
+----+-------------+--------+------+-----------------------+---------------+---------+-------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+-----------------------+---------------+---------+-------------------+------+--------------------------+
| 1 | SIMPLE | tablea | ref | PRIMARY,idx_tablea_nm | idx_tablea_nm | 53 | const | 1 | Using where; Using index |
| 1 | SIMPLE | tableb | ref | idx_tableb_id | idx_tableb_id | 5 | tablea.id | 1 | Using where |
+----+-------------+--------+------+-----------------------+---------------+---------+-------------------+------+--------------------------+
Let's do EXPLAIN
on the second one:
explain select * from tableb where a_id = (select id from tablea where nm = 'Alice');
+----+-------------+--------+------+---------------+---------------+---------+-------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+---------------+---------------+---------+-------+------+--------------------------+
| 1 | PRIMARY | tableb | ref | idx_tableb_id | idx_tableb_id | 5 | const | 1 | Using where |
| 2 | SUBQUERY | tablea | ref | idx_tablea_nm | idx_tablea_nm | 53 | | 1 | Using where; Using index |
+----+-------------+--------+------+---------------+---------------+---------+-------+------+--------------------------+
I don't have much data in those tables and with little data you will notice identical performance. As the workload changes, the execution play may change.