ASPX can be used with MVC instead of Razor (cshtml) although it is not common. So no, being ASPX doesn't automatically make it Web Forms, though the majority of time it will be.
MVC has the concept of pluggable View Engines. Note that just because you can use the ASPX extension does not mean that all the controls that you can use in Web Forms will be available in MVC. Everything still follows the MVC lifecycle, not Web Forms.
Note that having .cshtml doesn't necessarily mean you're using MVC. It could be Web Pages, Razor PDF or some other framework.