Working on texture upscaling, is there an easier way of extracting them?

C

capucine

Joined
Dec 27, 2018
I've been experimenting with using various AI-upscaling methods (and adding detail by hand) to make Shenmue look a little more up-to-date and have been making good progress. The way I've been doing it is with the SpecialK mod, which dumps out textures it finds in the scene that need a mipmap (which is the vast majority), then organising them and getting to work. It's getting to be a pain in the arse to have to explore the game manually and I often have to scan through every texture that's in the memory to find what I need. :( Also, SpecialK doesn't work at all for Shenmue 1 for me so I'm prevented from working on that completely. Are there tools to get at the packaged textures? I've had a poke around with ShenmueHDTools 1.2 but only found a few .dds files and I'm lost!

Some before and afters
http://oi65.tinypic.com/hwlc2q.jpg http://oi68.tinypic.com/epivqo.jpg
http://oi63.tinypic.com/t5593d.jpg http://oi67.tinypic.com/2z7qykx.jpg
http://oi67.tinypic.com/jgjndz.jpg http://oi63.tinypic.com/2ytz9s3.jpg
 
I wrote a tool with PhilYeahz called wudecon which will allow you to extract all models, textures, audio and everything else in the D3T archives.

You can then use Forklift by Raymonf to replace the textures, instead of using the SpecialK stuff.
 
Astounding work! I was wondering if this was possible when I first heard about the news of AI upscaling was a few weeks back (being used by some devs + fan-related projects for some older jrpgs) the results look spectacular to me (whilst remaining really faithful to the original assets).

The prospects of AI scaling in Shenmue 1&2 seem very promising and the fact that it could be a reality much sooner then I (originally) thought is exciting!
 
LemonHaze said:
I wrote a tool with PhilYeahz called wudecon which will allow you to extract all models, textures, audio and everything else in the D3T archives.

You can then use Forklift by Raymonf to replace the textures, instead of using the SpecialK stuff.
Click to expand...
Niiiiice, I think I've got all of Shenmue 1 textures unpacked! I'll have a mess about then see if I can get them back in the game with Forklift. I'm so excited :D
 
Awesome work. I previously delved in light AI upscaling with Ryo's jacket and jeans. Would be amazing seeing it in a larger scale.
 
Might you be able to attach the before and after screens, please? Links aren't working for me :/
 
Thanks much! Looks really nice. Jealous these can't run on consoles.
 
capucine said:
Sure thing, nothing mind-blowing, just what I could get a before and after of. Added a shot of Passport Ryo's texture WIP (ignore the bandage, that area's UV mapping is super warped and needs more work)
View attachment 1812View attachment 1813
View attachment 1814View attachment 1815
View attachment 1816View attachment 1817
View attachment 1818
Click to expand...
Hi! I registered on this forum just to ask you if there is any update on you project :sweatsmile:?? I searched for your work on youtube, steam, reddit and nexus mod and found nothing.

You were doing the god's work dude! I'm not a fan of those HD texture mods were people replaces the original textures with some real life grass, wood and stone photos, all the HD texture mods so far fail to remain true to the original visuals and artistic Yu Suzuki vision.

Some months ago I learned about those "Neural Networks/Machine Learning AI Upscaling" tools like Topaz Gigapixel and ESRGAN and was surprised about how nobody on the Shenmue community (which is one of, if not the most, dedicated) had made it yet, while games like Star Wars KOTOR, Doom and Final Fantasy VII already have... I mean, I don't have the capacity to do it myself as I'm not a modder, but I'm surprised that people who are capable haven't tried it yet.

The exemples you posted above are looking beautiful and are exactly what I was hoping for. In my opinion, upscaling the original textures with these AI-upscaling methods are the perfect way for having the true ultimate HD remaster this masterpiece deserves.
 
Work paused as I'm dealing with family illness now and free-time and general enthusiasm has withered. The method I was using at the time to get the textures in-game (SpecialK) is really not going to work for full-scale replacement. It absolutely chugs with a whole scene using new textures, and having every single texture loose inside a folder is unworkable (plus it's an unreliable buggy mess imo and I outright can't get it to work with Shenmue I). I got as far as extracting straight from the game files but the structure is difficult to work with; every model is packaged with its own textures, and every scene is its own file as well, which means so, so many duplicates of models and textures (and I'm not tech-savvy enough to cook up a little program to seek out these duplicates and automatically replace with my new textures, lol. They have generic names so I can't do a simple file name search. Maybe file hashes? idk). I didn't get as far as figuring out how to repackage the game files along with new textures :(
I am NOT giving up though, rest assured. The folders are still open on my desktop, lol. As soon as I get through personal life stuff I'm going back to work on this. Shenmue deserves it!
 
Last edited:
capucine said:
Work paused as I'm dealing with family illness now and free-time and general enthusiasm has withered. The method I was using at the time to get the textures in-game (SpecialK) is really not going to work for full-scale replacement. It absolutely chugs with a whole scene using new textures, and having every single texture loose inside a folder is unworkable (plus it's an unreliable buggy mess imo and I outright can't get it to work with Shenmue I). I got as far as extracting straight from the game files but the structure is difficult to work with; every model is packaged with its own textures, and every scene is its own file as well, which means so, so many duplicates of models and textures (and I'm not tech-savvy enough to cook up a little program to seek out these duplicates and automatically replace with my new textures, lol. They have generic names so I can't do a simple file name search. Maybe file hashes? idk). I didn't get as far as figuring out how to repackage the game files along with new textures :(
I am NOT giving up though, rest assured. The folders are still open on my desktop, lol. As soon as I get through personal life stuff I'm going back to work on this. Shenmue deserves it!
Click to expand...

[GUIDE] Replacing textures using d3t's SDTextureOverride

PhilYeahz and myself recently worked on writing some tools to aid with texture replacements using d3t's own SDTextureOverride.json format. The process is pretty simple. Extract both the attached archives to a folder Create a new folder within the folder you extracted the archives to and place...
www.shenmuedojo.com www.shenmuedojo.com

Using this method, I was able to make replacement texture mappings for every single texture in the game and then batch upscaled every texture and dumped them all into a single ~4GB modded TAC file. Results are much better than SpecialK, and you'll get support from me if you run into any issues. Sorry to hear about your situation, hope it gets better soon :).
 
capucine said:
Work paused as I'm dealing with family illness now and free-time and general enthusiasm has withered. The method I was using at the time to get the textures in-game (SpecialK) is really not going to work for full-scale replacement. It absolutely chugs with a whole scene using new textures, and having every single texture loose inside a folder is unworkable (plus it's an unreliable buggy mess imo and I outright can't get it to work with Shenmue I). I got as far as extracting straight from the game files but the structure is difficult to work with; every model is packaged with its own textures, and every scene is its own file as well, which means so, so many duplicates of models and textures (and I'm not tech-savvy enough to cook up a little program to seek out these duplicates and automatically replace with my new textures, lol. They have generic names so I can't do a simple file name search. Maybe file hashes? idk). I didn't get as far as figuring out how to repackage the game files along with new textures :(
I am NOT giving up though, rest assured. The folders are still open on my desktop, lol. As soon as I get through personal life stuff I'm going back to work on this. Shenmue deserves it!
Click to expand...
Oh damn, I hope everything is going well with your family illness problem! Sending positive vibes to you

I'm no expert but it seems that in the current version of the Shenmue HD ModTools 1.3 you can import textures by looking for pkf files... but I may be wrong ?

Also I don't know if it helps but I found this tool on Nexus Mod: https://www.nexusmods.com/shenmue/mods/22 : "Tools for extract and convert textures in ShenmueHD". This guy explains how to do it: https://www.shenmuedojo.com/forum/i...ure-editor-ipac-browser-update.568/post-14736

This may help too: https://www.shenmuedojo.com/forum/i...wudecon-file-model-converter-for-sm1-sm2.885/ (read the posts of @Boulotaur2025 and @ReeceMix in this thread, they are talking about extracting textures and resizing them)

And last but not least, it seems that @Streamer did exactly what you were trying to do with SpecialK and extracted every texture from the game: https://www.nexusmods.com/shenmue/mods/26 . If I'm not wrong you can download this texture pack, make your magic using ESRGAN or another AI-upscaling method and then use d3t's SDTextureOverride to replace your upscaled textures in the game!
 
Last edited:
LemonHaze said:

[GUIDE] Replacing textures using d3t's SDTextureOverride

PhilYeahz and myself recently worked on writing some tools to aid with texture replacements using d3t's own SDTextureOverride.json format. The process is pretty simple. Extract both the attached archives to a folder Create a new folder within the folder you extracted the archives to and place...
www.shenmuedojo.com www.shenmuedojo.com

Using this method, I was able to make replacement texture mappings for every single texture in the game and then batch upscaled every texture and dumped them all into a single ~4GB modded TAC file. Results are much better than SpecialK, and you'll get support from me if you run into any issues. Sorry to hear about your situation, hope it gets better soon :).
Click to expand...
Would you be willing to share your upscaling mod? I have been trying to do the same thing for the past few days and have been running into issues. For instance, I have all of the textures extracted via wudecon, and they are all pngs. Correct me if I'm wrong, but with the sdtextureoverride method I would have to convert them all to .dds? Must I do this with the forklift method as well?
 
zidane said:
Would you be willing to share your upscaling mod? I have been trying to do the same thing for the past few days and have been running into issues. For instance, I have all of the textures extracted via wudecon, and they are all pngs. Correct me if I'm wrong, but with the sdtextureoverride method I would have to convert them all to .dds? Must I do this with the forklift method as well?
Click to expand...
Could you share the textures you extracted via wudecon so @capucine could use his AI-upscaling method to upscale them? For what I understood he was having problem to extract the textures, so if you share yours, he (or everyone capable of doing it and with time and disposition to do it) would be able to upscaling it and everyone would be happy :D
 
Hi!
I would like AI-upscaling all the texture. I dont achieve to extract them.
Can someone share them here ?
 
zidane said:
Would you be willing to share your upscaling mod? I have been trying to do the same thing for the past few days and have been running into issues. For instance, I have all of the textures extracted via wudecon, and they are all pngs. Correct me if I'm wrong, but with the sdtextureoverride method I would have to convert them all to .dds? Must I do this with the forklift method as well?
Click to expand...
Hi!
I would like AI-upscaling all the texture. I dont achieve to extract them.
Can someone share them here ?
 
You must log in or register to reply here.
Back
Top