Your instinct was right - milk is roughly 87% water, so a same-quantity swap works and the dough behaves almost identically. What you noticed is exactly what changed: milk brings fat, sugars and proteins that give a softer crumb, gentler golden crust and a richer taste. Water gives a leaner, slightly chewier loaf with a crisper crust - not worse, just leaner.
The professional trick when you want milk's benefits with water's convenience: keep water as your liquid and add milk POWDER at 5-10% of the flour weight into the dry ingredients. Same softness and flavour, no fridge dependency, and it's exactly what a lot of commercial bakeries do.
So: 1:1 swap in either direction, expect the texture shift, and keep a pouch of milk powder for the days the fridge betrays you.