Java factory has little (any?) choice, but focus here is c++ factory techniques. I think it usually has to manufacture on heap.
In specialized cases, my factory can maintain a private pre-allocated object pool, carved out from heap, so the factory returns objects from this pool, without calling new().
To my surprise, some c++ factories actually return by value, as illustrated in both [[c++cookbook]] and https://herbsutter.com/2013/05/30/gotw-90-solution-factories/ RVO and move-sematics would likely kick in.