Somehow, I always end up building factories facing the wrong side (inside the base) and everything that's not flying has trouble getting out of the crowded nuke-magnet. I am suggesting arrows that show the default exit direction.
For some reason I always get confused with the advanced vehicle factory in particular. Its side walls look a lot like its ramps in wireframe; some arrows projected onto the ground in front of the exits would help clear that up without forcing art changes.
Give the factory a move order. Units will be built facing the closest exit ramp to the first factory move order and will roll off in that direction.
DidnĀ“t worked for me, my land factorys only opens the ramp in one direction. Tryed it several times to give them a new rally point after production, it will only open at one side so far. I just try to keep in mind that the 3 chimney must be on the right side of my land factory, in that case the doors will open to the south if the 3 chimneys are right. (if you look top down on the basic land factory) Rallypoint itself works fine (and yeah i tryed it with the "move" order too)
several factorys have two ramps- some can be identified easily, others not so much. setting rally points doesn't work. Submarines, ships, vehicles and bots don't use their reverse when deploying from factory.
When a rally point is set before a unit begins building, that unit will begin building facing the exit that has the shortest pathfinding path to the rally point. If it's already started building, it won't reverse out; I don't think units are presently even capable of reversing.
This has been my experience as well. Just set the rally point before a unit begins constructing and it will start oriented towards the closest ramp to the rally point. With regards to the OP, it sounds like you're building your base buildings to close together. Spread them out by just a little bit to help your units navigate.
I've noticed they work much better if you make your factory's waypoint start with one right in front of the factory (like a half-factory distance out) and then to where you want them to gather. Makes sure they're away from the building before they do anything complicated like turning.