I think you'll get the whole idea the moment you put yourself in SEGA's shoes - before the end of the Dreamcast, in the software period and after Sammy takes over.
Shenmue is a project that does not end with one game. It is successful until the time when the SEGA are done with the home consoles. Plans has changed in the "Software" period. Priority is given to easy-to-make games that will bring SEGA out of bankruptcy. After Sammy's take over, the priorities are copying successful models, no risks and following trends. Perhaps because of this, the idea of a Shenmue MMO game was initially nurtured, but then canceled when they saw that it wouldn't be easy and cheap. (even with outsourcing)
Shenmue I is a succesful start of a new series. It showcases the true capabilities of the Dreamcast and is a top best seller for the console. Shenmue 2 has no marketing whatsoever and has been practically destroyed by Software SEGA and their deal with Microsoft. Plans have changed. After that Sammy-SEGA practically freezes real Shenmue games develoment.
When someone makes comparisons between Yakuza and Shenmue they should keep in mind that Shenmue was an unfinished project that if it had been completed would most likely turned a profit. Yakuza is practically a finished project, from which similar games come out every year like FIFA.
Claims like "Shenmue 1 and 2 cost that money and earned that money" are ridiculous in the grand scheme of things. The goal of the game was to be a blockbuster that showcased the console's capabilities. The money invested in Shenmuе could be returned in many other ways if the Dreamcast was a successful console. SEGA are not stupid not to know that investments in such a project will not return immediately. Above we talked about how important prestige games and art games are for a new console in the market. Keep in mind that back then, SEGА didn't know they had no chance against Sony no matter what they did. :)
Why do you think that Yakuza wasn't successful? If the reported numbers (that it cost $21M to make and sold 1M units) are anything close to correct, then that's a success (not a crazy success, but definitely something that would earn a sequel). And why would Sega lie and keep funding it if that weren't the case?
Don't you think it could be a good image advertisement for the real business of the company? (pachinko, gambling, adult entertainment) :) Just like Shenmue is an advertisement for Dreamcast.
It's very straight-forward thinking that every game must necessarily return all the money immediately. Maybe a lot of money went into the engine and technology . If dreamcast hadn't been shut down, there would have been at least 3 shenmue games with this engine by 2004. Or maybe they will use it for samurai shenmue side-game in the past too!
It is enough for Yakuza to recoup the investment in the first few games (1-4) . Yakuza series has extremely high-quality main and side games. It uses assets from the old games and is very well thought out as a successful commercial product. After 10 years and 10 games, the series starts making money in the west as well.
Long story short, Shenmue project was a success until it was shut down prematurely for completely different reasons.