LAMMPS distributes the work by performing domain decomposition. For example, with spatial decomposition, the simulation volume is cut into boxes and all particles in a particular box are assigned to some MPI rank for processing. Because integration in molecular dynamics is a globally synchronous operation, each step completes only then when all MPI ranks are done. Thus, the time it takes to complete an integration step is equal to the time it takes the slowest MPI rank to complete its assigned box(es).
M1 is an architecture similar to ARM DynamIQ (successor of big.LITTLE), packing both cores that are fast but power hungry (Firestorm cores) and cores that are slow but power efficient (Icestorm cores). The M1 chip in Mac Mini has four of each type. Since macOS does not provide explicit CPU affinity, the MPI library simply launches the given number of ranks and it is up to the OS to schedule them as efficiently as possible on the available cores. With up to four MPI ranks, they all end up on their own Firestorm cores. Once you get past four MPI ranks, some will either land on one of the slow Icestorm cores or will timeshare a fast core with some other rank. In both cases, there will be at least one rank that will be much slower than the rest and hence the overall performance will suffer.
Another thing to note is that modern CPUs vary their core frequencies based on the thermal conditions. If only one core is loaded, it may be boosted way over its nominal frequency. When more cores are loaded, more heat is produced and the cores will not get as high boost in frequency as in the former case. Therefore, going from 1 to 2 to 4 cores will not give you linear speed-up, even if the problem is embarrassingly parallel.
Also, keep in mind that the kind of domain decomposition used can have an enormous effect on the parallel performance. For example, with spacial decomposition, if the atoms are not equally distributed among the subdomains or if they move between them in a non-uniform way, there will be load imbalance and again the speedup will suffer.