Development Log #2

Joined
Jul 27, 2018
Hello and welcome to today’s entry in the development log for Shenmue: Dragon & Phoenix Collection. My name is Dewey, and today, we’re going to be discussing the audio in Shenmue.


Now, as with any game, audio such as music and sound effects play a large role in the general atmosphere the game’s world provides. This is no different in Shenmue, and in the opinion of many, Music in particular is a primary focal point of how Shenmue makes its game world feel vibrant and alive. From the smooth jazz of Heartbeats, to the lighthearted jingle emanating from the Tomato Store, the Shenmue games really do seem to go out of their way to place the soundtrack in the forefront of the atmosphere; and we intend to do no different here. A tall order, to be sure, but one I am certain we can fulfill, so let’s all take a few minutes, to discuss and understand exactly how we intend to handle this in Shenmue: Dragon & Phoenix Collection.


Music in Shenmue, for the most part, is encoded in a proprietary format, that makes the most of the AICA sound processor present in the Sega Dreamcast. It is a MIDI based format, making use of samples compressed to various degrees, along with other features provided by the AICA processor, such as panning and multichannel mixing, to present the almost CD-quality soundtrack present within the game. The D3t releases does in fact utilize the same format, however for reasons that are not fully understood, it appears they weren’t entirely certain how the format and/or AICA processor worked, and as a result, had to alter these files to facilitate playback, leading to inaccuracies. This is noticeable in some areas more than others; and while it is overall a rather small detail in what is otherwise a generally faithful port, it does tend to stick out like a sore thumb given the prominent nature of music in creating Shenmue’s unique atmosphere. As such, I’m happy to say that as a result of a collaborative effort between Lemonhaze and team friend Sappharad (https://github.com/Sappharad), we now fully understand the changes D3t made to these files, can reverse them to be 1:1 with the Dreamcast releases, and are now able to accurately play back these tracks through Unreal Engine 4. To illustrate this, let’s take a moment to listen to one of these tracks, and compare between the Dreamcast and PC releases, and then let’s take a listen to what it sounds like in Unreal Engine 4. Today, we’re going to listen to Queen’s St. from Shenmue II; I believe this to have been a particular sore spot for many players in the rereleases, so I feel this should be a good illustration as to the level of accuracy and authenticity we are striving for, particularly in regards to the Dragon Edition portion of this project.


Dreamcast


PC


Unreal Engine 4


Now, this is all well and good for Dragon Edition…. But what about Phoenix Edition? What about our audio upgrades? Well, my friends, I have good news for you. I’m happy to announce that we are working with Richard Cartlidge, who has been diligently working on recreating a large variety of songs from the Shenmue soundtrack. We’re very excited to have Mr. Cartlidge on the team, and have the utmost confidence that he can revitalize the soundtrack and make the soundtrack to Shenmue: Phoenix Edition every bit as entrancing as the original. Now, being that this partnership is only a few days old, we don’t have much to show to this end just yet. However, you can check out the work he has already done on Youtube here:
here
https://youtu.be/oPxt3_9ygPQ

Of course, this now leaves the elephant in the room. Dialogue. Unfortunately, at this time, there is nothing to announce in the way of enhancement for dialogue audio for Phoenix Edition. As it stands, there is simply too much, and the quality of it is too varied, for any of us currently involved with Shenmue Dragon & Phoenix Collection to do much in the way of true enhancement. AI upscaling of audio is a very involved process, doubly so when speech is involved. With where we are now, it would require a separate, dedicated team to go through the dialogue, and process it to a point where it would be worth using, not to mention we don’t have anyone on hand with experience in AI audio upscaling. Even then, results are not guaranteed, and are very limited by current AI audio restoration technology. This regrettably means we have to relegate this to the “someday” pile. Other ideas have been suggested to us, such as using dialogue from Shenmue: The Movie (which is, notably, the highest quality source we have for some dialogue), but unfortunately, this also has its own drawbacks. In order to use this as a source for dialogue, the movie’s audio must be extracted, re-cut, re-timed, and then processed to remove background noise such as music and sound effects. This introduces its own artifacts, and it is simply a matter of personal taste as to which is more pleasant to listen to; the compression artifacts from various releases of the game, or the editing artifacts from covering up background noise in the movie’s extracted audio. For this project, I firmly believe that the former is the better option here, as not only do I find the compression artifacts less intrusive, Shenmue: The Movie only accounts for a small fraction of the overall dialogue present within Shenmue and Shenmue II. Introducing these other artifacts, as well as the generally enhanced quality of these lines of dialogue, will not only highlight both types of artifacts, but will equally make the disparity in quality stand out even further. For these reasons, the current plan is to leave the dialogue audio in place as is for the time being, in the interest of creating a more consistent experience. That being said, as we do intend for Shenmue: Dragon & Phoenix Collection to have mod support, there is absolutely no reason why a separate team could not create an enhanced dialogue package after release. Theoretically, one could even begin work on such a package now, as all the tools and documentation available (on the Shenmue side of things) are readily available, however it is just not feasible for us to undertake such an endeavor at this time.


This news, however, is not all bad. There is a silver lining that I hope we can all take solace in. This move to Unreal Engine 4 affords us a lot of flexibility in terms of visual fidelity. We can take these old models and textures, and slap a new coat of paint on them. Truly make the best out of these assets made before the turn of the century. This flexibility, however, also extends to audio. Modern engines have a host of audio features that allow us to truly make the best out of a bad situation here. Unreal Engine 4 natively supports spatial audio rendering, which in and of itself goes a long way towards covering up the hiss of audio compression, but atop that, we have a bevy of options towards the attenuation of these sounds, allowing us to really bring out everything we can from these ancient recordings.


Well, that’s about all I have for this entry. I’m sorry the news couldn’t be all good; but as always, we’re open to suggestion, so feel free to stop by our Discord and we’ll be happy to discuss whatever it is you have on your mind.
https://youtu.be/_VL7A8ymVVs

Until next time,

Dewey
 
Last edited:
Joined
Aug 19, 2018
I just love the collaboration and effort of the team here. Honestly while my mind is racing with the possibilities the Phoenix edition will bring I am just happy to see complete faithful port to UE4 and the nice upgrades that come with that.
 
Last edited:
Joined
Aug 26, 2018
Location
Spain
Great writeup! I personally love this kind of technical buy easy to understand texts. If you keep doing them it will be another gift from this wonderful project,

One question out of curiosity, for the music playback in Dragon edition, have you written an AICA interpreter that will play the tunes in real time? Or the idea is recording versions of the AICA inhouse and then play them as a prerecorded "streaming audio" file in game?

If I'm not mistaken, the d3t rereleases did the former (and of course the Dreamcast itself). But from the video you posted of UE4, it looks like the latter. Both of the approaches would be good, anyway.

Also glad you got Richard James Alexander Cartlidge on board. I'm suscribed to his youtube channel and all his material is top quality. He could endup being the Tee Lopes of the Shenmue franchise :-)
 

Kenzo38130

"Youngest shenmue fan"
Joined
Nov 11, 2019
Location
France
Favourite title
Shenmue II
Currently playing
Hood Story: Kaito Yamazaki
PSN
Rdc433
i can't wait
 

Tentei

ライトニング
Joined
Jan 31, 2021
Favourite title
Shenmue II
PSN
miraigaara
The fact my recent finishing of replaying part 2 and the feeling it left me with and knowing this is happening is the best thing for us fans until we get four. I am so happy I got introduced to this game when I did. its been one hell of a ride! GOOD JOB EVERYONE!
 
OP
OP
D
Joined
Jul 27, 2018
One question out of curiosity, for the music playback in Dragon edition, have you written an AICA interpreter that will play the tunes in real time? Or the idea is recording versions of the AICA inhouse and then play them as a prerecorded "streaming audio" file in game?


We are doing the former. In the video, before playback starts, if you look on the right hand side of the screen, I highlight a box that says "0017_020.SND". This is pointing Unreal Engine towards the actual .SND file from the games, not a recording of it.
 
Joined
Aug 26, 2018
Location
Spain
We are doing the former. In the video, before playback starts, if you look on the right hand side of the screen, I highlight a box that says "0017_020.SND". This is pointing Unreal Engine towards the actual .SND file from the games, not a recording of it.
Incredible!
 

Kenzo38130

"Youngest shenmue fan"
Joined
Nov 11, 2019
Location
France
Favourite title
Shenmue II
Currently playing
Hood Story: Kaito Yamazaki
PSN
Rdc433
Well what fun is it if we just tell you? Look closer.
i found it , i asked about the link because i dont speak english very well so i didnt understand what op mean
 

Kenzo38130

"Youngest shenmue fan"
Joined
Nov 11, 2019
Location
France
Favourite title
Shenmue II
Currently playing
Hood Story: Kaito Yamazaki
PSN
Rdc433
Is that some bloom effect or is that just to much gamma? ;P
(Sorry I couldn't hold back myself, I just don't like bloom shaders.. mostly I guess).
It would be cool to keep colours and contrasts of the original.
They probably use post processing effect for the game i like it personally it look more realistic
 
Joined
Aug 26, 2018
Location
Spain
Anybody else found the hidden links in the OP yet? 👀
Hadn't done until you gave us the clue...Awesome stuff! I understand how the team is reusing the originals data about camera positioning, for example.

(I guess it's a Phoenix edition thing for the snow pouring inside the dojo :p)
 

BruceWayne911

Upper Knuckle (⬅️ ⬅️ 🤜)
Joined
Jul 29, 2018
Favourite title
Shenmue II
Currently playing
Shenmue 3 & Yakuza Kiwami
This is cool!- Ryo Hazuki
 
Top