One thing to note here is that models are not supposed to extend from factories, instead you would normally use the factories directly, which in turn should return a model initialized. E.g. imagine you have a frozen pizza (model) and a oven (factory); rather than eating the frozen pizza directly, you use the oven first, from which in turn you get the actual edible pizza. For a very complex type of oven (read, if the object creation process involves a lot) the factory pattern makes sense.
Other than that, no reason not to use the pattern if you feel it is useful for your needs. I generally include the factories directly where I store the models aswell, since they both strongly depend on each other (as in, one would not work without the other), but that's just a matter of taste I guess.
One last tip: use patterns when you really think you need them, not because they sound fancy. If you think not using the factory pattern makes your life easier and will get your project done faster, then by all means don't use it. That's not to say you shouldn't explore new patterns, but always rationalize whether the time and effort to implement and maintain them is justified.