Gens4All with Z80 Emulation

Place for discussing homebrew games, development, new releases and emulation.

Moderators: pcwzrd13, deluxux, VasiliyRS

TapamN
letterbomb
Posts: 148

Gens4All with Z80 Emulation

Post#1 » Wed Aug 18, 2021 1:59 am

Earlier I posted a version of Gens4All with PVR accelerated VDP emulation. I wasn't able to get Z80 emulation to work, so most games had problems with sound. I recently went back to try to get it working again.

I tried using Neo4All's Z80 core in Gens4All, but it didn't work. I decided to try to look through the differences between Gens4All and Neo4All's Z80 code to see if there was anything that might help.

I noticed right away that some compile options were different between them. I replaced Gens4All Z80 core with Neo4All's and changed the options back the Gens4All's settings then tried out the result.

The Z80 emulation now worked. Games that used the Z80 for sound now had working sound.

I was expecting to have to do more than that, but I'm not complaining.

With the overhead of Z80 emulation, games had more trouble staying at 60 FPS. After disabling the Gens4All profiler, the speed was actually better than before. (Running the profiler added to Gens4All took more time than emulating the Z80) I was targeting 22khz sound at 60 FPS without frame skip, but with some games you can almost do 44khz. Others sometimes have trouble at 22khz. In Thunder Force 2, during the "Good Luck" screen when starting the game, there's heavy slow down for some reason.

There were problems with random freezes. This was especially common with the Sonic SMPS Z80 games (Sonic 2, 3, &K, 3D Blast) but I had it happen to me once in Vectorman (GEMS). I already had added a watchdog thread to return to dcload if the main thread seemed to freeze. I modified the watchdog to print information about the state of the 68000 and Z80. The Z80 was set to run for over a billion of cycles in once shot, when it shouldn't run for more than a couple thousand before Gens gives the 68000 a chance to run. I figured out the large cycle time was coming from reading past the end of a lookup table, Z80_M68K_Cycle_Tab, used when calculating how many Z80 cycles are lost when the Z80 gets disabled by the 68000 (so that the 68000 can access Z80 RAM). I increased the size of Z80_M68K_Cycle_Tab, and, as an extra measure, modified the table look up to wrap around instead of reading past the end of the array, which should prevent these freezes.

There's still the question of why PC Gens was able to get away with the size of the table it had, and Gens4All couldn't. (Also, I noticed that main RAM is allocated after the cycle table. So when it was reading past the end of the table, it was reading from Genesis main RAM and using that as timing information. I though that was interesting.)

Ristar (using a modified SMPS 68K driver) would freeze during the Sega logo. It looks like the 68000 was stopping the Z80 and reading a byte from Z80 RAM. If that byte was nonzero, it would resume the Z80, wait a bit, then try again. I guess a nonzero byte indicates that the Z80 is busy, so the 68K was waiitng for the Z80 to free up, but the Z80 wasn't clearing it. For Ristar, I added a hack that causes 68K reads from that byte to always return 0. It's enough for Ristar to be playable, but there are occasional incorrect or missed notes and sound effects.

Another issue is that the Sonic SMPS Z80 games had music tempo issues. It was like the Z80 was slightly drunk, and had trouble keeping to the beat. I found a workaround when fixing the Z80 table problem; by changing it so that the Z80 does not lose time when the 68000 accesses Z80 RAM, the music timing was fixed. I noticed this hack would cause Streets of Rage 2 to freeze on certain music tracks, like Ristar did, so the hack is only enabled on Sonic SMPS Z80 games.

All these problems probably originate with the Z80 emulation. There's likely an error in the timing emulation somewhere. The correct thing to do would be to find where in the Z80 code the error is, but these hacks should work for now.

Phantasy Star 4 had slowdown in certain areas. It didn't seem like the SH4 was overloaded, but it ran at half speed. Eventually, I noticed in one place the slowdown would occur only when the press-button-for-more-text arrow was not visible (?!). This seemed like an issue with the PVR rendering code, like something invalid up was being sent to the PVR and it was choking on it. When PS4 was having slowdown, the PVR was taking about ~20 ms to render the frame. I couldn't find anything that looked wrong with the rendering code, so I tried switching from drawing each tile as a pair of triangles to a single PVR sprite and switching to using the punch-through list instead of the transparent list, and the changes reduced GPU load enough to hit 60 FPS. I guess sorting the triangles was too hard somehow? Or maybe the way I converted VRAM to a texture was a problem? (I was using triangles and transparency because it would have made it easier to add support for shadow/highlight and certain types of raster effects.)

Known issues:
  • Some games have trouble with consistent 60 FPS
  • Sometimes there's popping in audio. Noticeable in the left channel of Phantasy Star 2's title screen music.
  • Requires hacks for Sonic 2, Sonic 3&K, Sonic 3D Blast, and Ristar sound drivers (Enabled automatically by checking title in game header)
  • 256 pixel wide screen mode is left aligned, and offscreen sprites are sometimes visible on right
  • Tilemap errors on the edges of screen
  • Shadow/highlight doesn't work
  • Raster effects aren't supported at all
  • Line scroll is treated as cell scroll
  • No vertical cell scroll at all
  • Flickering in Out of this Word/Another World
  • Sonic Spinball demo desyncs

Possible improvements:
  • Optimize VDP renderer (It currently takes about 3 ms, can probably be improved to 2.25-2.5 ms, which would give more CPU time for future VDP accuracy improvements)
  • Support for more than one controller
  • Six button/mouse controller support
  • Fix edge of screen scrolling
  • Center screen when using 256 pixel wide mode and hide off screen sprites (Not sure how possible fixing the aspect ratio would be without shimmering on horizontal scroll or blur)
  • Support PAL50 output for PAL games when not using VGA
  • Better line scroll support (At least get it up to Smash Pack level)
  • Shadow/highlight mode
  • Interlaced mode (Sonic 2/Combat Cars splitscreen)
  • Support for a single full-palette color change raster (Water line in Sonic games)
  • Screen on/off raster (Used for letterbox in some games like Gunstar Heroes opening)
  • Background color change raster support
  • Vertical cell scroll without horizontal cell/line scroll (Might help parts of MUSHA?)

Probably not possible:
  • Getting raster-based driving games to display correctly (Out Run, Road Rash)
  • Fixing all priority errors
  • Correct line scroll with vertical cell scroll
  • Perfect raster effects

Even with it's limitations, this emulator is still a huge improvement over Sega's emulators and previous versions of Gens4All I've tried, and there are plenty of games that run well enough to be enjoyable.

The attached file contains the new Gens4All source code.

Last time I released what I had, some people had trouble getting it to compile, because the 68K core required GLib. I left the object files in, so if you don't change the 68K emulator or "make clean", you should be able to compile Gens4All without GLib. I used GCC 9.3 to compile this. If you're using an older compiler, these object files might not work. Compile with "make -j 8 -f Makefile.dc".

There's no ROM loader. You have to modify main in main.c to specify what ROM to load through dcload. Find the line with "ROMDIR" define and change it to point to where your ROMs are stored, then change the following run line to the name of the ROM.

Controls are:
Genesis C = DC A
Genesis B = DC B, X
Genesis A = DC Y
Genesis Start = DC Start
Genesis D-Pad = DC D-Pad
Exit = L or R

There are extra keyboard controls, for debugging, but they aren't worth listing here.

No video since I'm not in a position to use my capture device.
Attachments
gens4all.7z
(1.26 MiB) Downloaded 570 times
Last edited by TapamN on Wed Aug 18, 2021 2:14 am, edited 1 time in total.

User avatar
Ian Micheal
Developer
Posts: 5980
Contact:

Re: Gens4All with Z80 Emulation

Post#2 » Wed Aug 18, 2021 2:10 am

Amazing work :) and all the info on how you did it :) Yes i had trouble compiling it last time i even went as far as to build whole new tool chain with linux mint but was still a problem :) thank you :D


It's also great seeing more development of older projects and emulation on dreamcast it's what drew me to port and do things with the machine in the first place..

Looking forward to your new pvr driver one day :) :geek:

I could write simple rom loader if you wanted but I'm sure you could do that..

User avatar
FlorreW
Animated Violence
Posts: 499

Re: Gens4All with Z80 Emulation

Post#3 » Wed Aug 18, 2021 5:18 am

Great work mr TapamN.

Thanks alot , and as Ian says , always nice to see also the old projects get some love

User avatar
dubcity
Super Sonic
Posts: 1575

Re: Gens4All with Z80 Emulation

Post#4 » Wed Aug 18, 2021 5:34 pm

Very detailed write up. Good work. Thanks for your contribution.

User avatar
soniccd123
dark night
Posts: 63

Re: Gens4All with Z80 Emulation

Post#5 » Wed Aug 18, 2021 8:34 pm

Could you explain what is needed to build it? Would like to test it :)

MastaG
Quad Damage
Posts: 203

Re: Gens4All with Z80 Emulation

Post#6 » Thu Aug 19, 2021 12:04 pm

@TapamN
Much love and respect for keeping on with your improvements.
Personally I think both the SNES and Genesis were the best consoles ever.
I will build your project soon and post some binaries if nobody else beats me to it!

TapamN
letterbomb
Posts: 148

Re: Gens4All with Z80 Emulation

Post#7 » Thu Aug 19, 2021 5:45 pm

I uploaded a video showing how it's currently working.



I looked into the clicking sounds I mentioned before, and it seems to be an issue with SDL. I dumped the raw data sent to SDL and played it back with Audacity, and there wasn't any clicking. So the issue seems to be with SDL. I thankfully won't have to try to debug the sound emulation. I tried replacing SDL with KOS streaming functions, but they don't seem usable. I had trouble getting them to work. Looking through the source for it, the implementation seems to require (but doesn't mention in the documentation) a buffer greater than 2048 samples, or about 1/10th of a second at 22KHz, for the streaming to work. I had the buffer set to about 1/50th of a second, and the streaming library thought that there wasn't enough to bother loading more samples. That minimum buffer size is way too big to be usable for an emulator. My next goal is getting a lower latency streaming replacement to work, without clicking.

Ian Micheal wrote:I could write simple rom loader if you wanted but I'm sure you could do that..

I was thinking of having a ROM list file, which would the user to customize the order of the list and store game settings. If the game needs any emulator hacks to work correctly, they could be specified in the list instead of trying to autodetect based on the ROM title, which would be useful for playing hacks. An entry might look something like this:

Code: Select all

Game: Sonic the Hedgehog 3 & Knuckles
Rom: s3&k.gen
Controller: 2p_3button_std
Samplerate: 22050
Emu: sonic_smps_z80_hack shadowhilite_spr
Savename: GENS4ALL_S3K
Savetitle: Sonic 3 & Knuckles
Savetitlevmu: Sonic3&K
Saveicon: s3kicon.bin

There could be multiple lists, so one might have commercial games, another might have hacks, or something.

I'm more worried about the emulation quality at the moment than getting any working loader, though. I'm not really good at graphic design. If anyone wants to come up with ideas for how it should eventually look, feel free.

soniccd123 wrote:Could you explain what is needed to build it? Would like to test it :)

All you should need, with the included object files, is a KallistiOS environment. If you want to make clean and build the source from scratch, the 68000 core requires GLib 2 to compile. You need the development files for GLib, not just the binaries. On Xubuntu, that should be "libglib2.0-dev".

I noticed KallistiOS recently got an update to the PVR driver that looks like it would break 3D under certain conditions. It fixes one memory leak, but mistakes confusing but correct code for a second leak. The vertex buffer and one part of the OPB end up overlapping, and that part of the OPB is set a a very small size. It probably wouldn't affect Gens4All, but if you have rendering errors, use an older version of pvr_buffers.c and recompile KOS.

User avatar
soniccd123
dark night
Posts: 63

Re: Gens4All with Z80 Emulation

Post#8 » Thu Aug 19, 2021 8:41 pm

Thanks!!, got to build it fine, just need to get my custom "coders cable" to test it. Just to ask, without a rom loader I supose that, for now, it won't work on GDemu right?

dcsteve
undertow
Posts: 27

Re: Gens4All with Z80 Emulation

Post#9 » Thu Aug 19, 2021 10:40 pm

Incredible. Never thought i would see an improvement over Black Aura and Warmtoe's Genesis Plus DC builds. This runs sonic 2 much better with no delays in sound and better quality sound too!

User avatar
Anthony817
Shark Patrol
Posts: 4009

Re: Gens4All with Z80 Emulation

Post#10 » Fri Aug 20, 2021 12:24 am

This is absolutely incredible seeing the Genesis emulator running full speed on a stock Dreamcast with correct sound! This is literally a dream come true! Keep up the amazing work TapamN!
Image

  • Similar Topics
    Replies
    Views
    Last post

Return to “New Releases/Homebrew/Emulation”

Who is online

Users browsing this forum: No registered users