Yes, I think that is right.
Jackson was probably much more influenced by his pre-war colleagues on the faculty of the Virginia Military Institute. They pretty much all went into the Rebel army. I recall reading some where there not a single VMI grad served on the Union side.