Brexit risk looms larger as UK banks get smaller