Wulinshu, a Shenmue hash database

Joined
Aug 11, 2018
Hi.

As some may know, the Shenmue re-release uses a hashtable to look up files it needs from the TAD and TAC files.
This makes modding very inconvenient, and means we'd need to capture file paths while the game is being played, because we have no way to get the real filenames and paths.
Until we have a somewhat large hash database, modding will be somewhat limited.
I made a hash database for this reason: https://wulinshu.raymonf.me/

If you'd like to volunteer to capture hashes, inject one of the DLLs to your game while you're at the main menu. Just play through, and it'll send the needed hashes to my server.
You can use Cheat Engine or any other DLL injector. For Cheat Engine, open the Shenmue process, click Memory View, and then use Tools > Inject DLL. It'll ask you for an import, so choose __v0.
You can also add it to your import table (or use a dx11 proxy that loads all DLLs in the Shenmue folder), which will cause the DLL to load at startup.

Shenmue 1: https://cdn.discordapp.com/attachments/481695424131104778/484419599014821899/WulinshuSM1.dll
Shenmue 2: https://cdn.discordapp.com/attachments/481695424131104778/484419559005356110/WulinshuSM2.dll
Source soon:tm:...

Regarding privacy, I store IP addresses of people who send in hashes. This is only used to "uniquify" results and calculate the "# matches" column. They will never be exposed to the public if I'm doing my job right.

The server runs a parse job every hour.

Thanks!
 
Last edited:
Can this be used to Capture the files it uses during gameplay , or does it just create a text/hash list?
I will be doing extensive file comparisons & model conversions this week DC vs PC
 
Can this be used to Capture the files it uses during gameplay , or does it just create a text/hash list?
I will be doing extensive file comparisons & model conversions this week DC vs PC
It creates a hash list and sends it to my server. The result is displayed on this website.

You can find the extracted files using the hashes.
 
Its a small issue but could you get the 'Path' to sort alphabetically ?
https://wulinshu.raymonf.me/#/

Also does anyone know of a program that will turn a file directory structure into a easy to read txt file?
I'm very familiar with DC file structures and files and I think I can rebuild the current model/texture structure, rebuilding PKF files is still a big issue when it comes to modding 'all' textures.
On the extracted ShenmuePC files it shows a 'model overide' file looking at the wulinushu list it shows that is a folder with contents, either way its all progress x
 
would love to help out with this but it wont work for me. once i inject the dll it asks to execute function. then i get this and game crashes
 

Attachments

  • failed.PNG
    failed.PNG
    91.3 KB · Views: 24
Is it normal that some errors appear? Second time game didn't crashed, but not sure that something was sent, at least I have not access to the web server.
 
Is it normal that some errors appear? Second time game didn't crashed, but not sure that something was sent, at least I have not access to the web server.
Are you using v1.01? The hook crashes if you use v1.01.

This is really useful... is there any chance of getting some kind of text download?
Well... not really. At least not for now. Having to generate large text files from this would be expensive.
 
I have 1.00. actually, checked it again and noticed that log appear in the cmd window. Paths can be read properly, which is good. But periodically html page appear with access error - is it ok, or you should add my IP to the white list?
 
I have 1.00. actually, checked it again and noticed that log appear in the cmd window. Paths can be read properly, which is good. But periodically html page appear with access error - is it ok, or you should add my IP to the white list?
Ahh, I see. Should be okay now. Could you try again?
 
No errors, guess it works now

So, someone should finish the game completely now with active scraper, right?
 
No errors, guess it works now

So, someone should finish the game completely now with active scraper, right?

Not just finish the game; visit every area, partake in every activity, talk to every person you can as often as you can (although that's possibly a bit too far). Basically we have to do EVERYTHING. Hence, crowdsourcing it.
 
I really think that the website should display how many % of the hashes we have completed. I'll be trying to figure out how many files total we have in order to do that calculation.

Also, I think this should be integrated into a future Shenmue modloader as some kind of opt-in "help modding" thing. Let's hope d3t doesn't decide to change the hashing algorithm or break the current mod methods.
 
The hook hasn't been updated for 1.01 yet (but the Shenmue 2 one should still work, I think).

We're working on a pseudo-mod loader right now. When that comes out, it'll have this hook integrated. So, until then...
 
why do you think same files appear with different hashes? I thought it may be different versions, but you said it will work only for 1.00
do you think versions will have not just different tac/tad names, but files hash database too? so it should be updated with each release.

what functionality this loader will have? as I am making a script that can enabled or disable selected files by clicking checkboxes, so there will be no need to invalidate anything manually. hope we are not doing the same.
 
The modloader allows you to place any modified files in a "mod" folder inside the Shenmue folder. You can keep the original file / directory structure, never needing to bother with hashes as long as you know the file path.

I've made progress towards loading files, and indeed, loading files with size <= original size works. However, files bigger than the original file in the TAC will fail to load since they won't be read completely (and the buffer allocated won't be large enough..)

I'm having a few issues with time right now, due to real life activities. But modding will be more convenient once me and Raymonf defeat the hurdle mentioned.

EDIT: Made some progress on that issue. Game doesn't crash now, but it's hanging.. The issue is specifically with Ryo, and I'm actually concerned that the mod I did on his MT5 is the issue, considering I could load a 16MB DDS texture for the watchface.
EDIT2: The mod isn't the issue. Welp. I guess something else is using the MT5's TAD size for something.. Raises a question about being able to do this other way.
 
Last edited:
Back
Top