I agree with your opinion for the most part. After Shenhua's explanation about Bailu Village in Shenmue II, I was under the impression that Bailu was an isolated village. In Shenmue III however, Bailu comes across more as a mountain town. It's one thing to have a few modern conveniences, it's another to have a modern arcade, gacha machines, and gambling stands. I do think however, that herb collecting, wood chopping, fishing, and the farmers market make perfect sense.
If anything, I wish Shenmue III expanded on these activities and dovetailed them into the narrative. You could add minigames such as hunting, making medicine, cooking, embroidery, building a home, on top of your martial arts training. As you do more for the village, the more events you trigger such as clearing out that landslide in Shenmue II.
As you complete more events, the more likely that key characters like Elder Yeh and Master Feng will reveal more history on the mirrors, Iwao's training, the prophecy, and so on. In other words, the more intimate a relationship you form with the village, the more they give back to you.
Bailu Village should have been a time for Ryo to truly heal, contemplate, and grow as a character. Sadly that was not the case as the game rushed us into Niaowu (Where admittedly arcade games, gambling, and gacha machines make more sense). And making Bailu feel more like a town instead of a village, made Shenhua feel more like an outlier than a natural resident.