They did get noticed though. The two guards battered them and they ran off. The thugs remained in the village the whole time you were there. There was no reason for those guards to guard the bridge the whole time.