A semaphore covers different use cases than a mutex.
With a mutex, you would be right. A mutex is typically used to prevent concurrent execution of critical sections in the code. A particular thread will acquire the mutex at the start of the critical section and release it when it leaves the critical section again. Having a mutex being released by a different thread than the one that acquired it would mean that the critical section is spanned across multiple threads, which is most probably not how you want to do things.
The use case for a semaphore is slightly different. It signals availability of a resource. A thread that needs to consume a resource will acquire the semaphore, which is conceptually acquiring the underlying resource. If no resources are available, the acquire will block.
Now, in the scenario where we talk about a fixed set of resources (like a set of available I/O ports) it makes sense for the acquiring thread to also release the resource again. I acquire the port, do some work and release it when I'm done so that other threads can do work on it.
But this is not the only use case for semaphores. Think of producer/consumer: A producer thread might provide resources (like items that are queued for processing by a worker thread) and the consumer thread will accept them. In these scenarios, consumed resources are typically gone, so you do not release your resources after acquiring them. Instead, the producing thread calls release to indicate that there is stuff available for consumption. The consumer then calls acquire to claim a produced resource and process it. A producer will never call acquire and a consumer will never call release.