that's it then, the final piece.
You can play games, record scenes, and boot games back up with the
recorded map.
It's not perfect, _far from it_, but it works!
There is still a lot of work to do, improving heuristics and such,
but I think wideNES is at a point where it can be merged into
master and shown off a bit :)
exciting!
wowie this was a trickyboi to figure out, but I think I managed to
get a decent-working solution.
The algorithm consists of 2 parts: 1) detecting scene transitions,
and 2) figuring out if the new scene is a duplicate.
Part 1) was solved using perceptual hashing, and watching the delta
of the frame hash between frames. When it spikes, odds are that's a
scene change.
Part 2) was a bit trickier, but boils down to hashing every frame
and associating it with a scene + scroll values. That way, when a
scene transition is detected, I can check the hash-maps to see if
the "new" frame is actually one I've seen before.
There is a bit of extra work that gets the technique working with
elaborate fade-in animations / animated backgrounds, but it's not
magic or anything.
What's left?
Well, 1) needs to be improved, since the current delta threshold
value i'm using has been pulled out of thin air, and while it gives
reasonable results on many games, ideally, it should be more robust
2) is pretty good though!
All that I need to do now to be super happy with it is serialize
the scene data and enable loading / saving it to disk! Once that's
done, the initial vision of being able to automatically map out a
game will have come true!
exciting!
Perceptual hashing... that's the cool shit.
I mean, my "advanced" perceptual hashing algorithm is really dumb
right-now, literally just taking a sum of all pixels in the frame,
and yet, it's good enough to detect when a _bunch_ of pixel-values
change from frame-to-frame. I bet that with a bit more tweaking,
like normalizing the frame wrt. the palette, I can get some even
better results.
I also have a "true" hashing implementation, where each frame gets
a unique hash. It seems to be realtively collision-free. That said,
there are times where there are collisions, namely in Metroid,
where the game likes to reuse screens. I'll have to figure out how
to deal with that...
So, what's the next step?
Whenever a scene changes, take the hash of the new screen, and see
if it matches up with anything we've seen before.
That way, we can have "re-enterant" scenes, which is a key to being
able to save/load tiles!
didn't think i'd be able to pull it off, but Zelda seems to scroll
fine now! All I needed to do was sniff the PPUADDR register, since
after all, PPUSCROLL and PPUADDR both point to PPU t/v anyways...
I also bunched-up the heauristics a bit better, hopefully making
the code a bit nicer.
before showing off my work, the big thing that needs to get done is
scene detection. Yeah, heuristics are cool, and fun to implement,
but I can do that later. What I _should_ focus on is getting scene-
detection working, and afterwards, saving and loading, since that's
the killer feature (imo)
It's going to be tricky, but i've been reading-up on perceptual
hashes, so that might be useful?
Alternatively, I could sniff what games tend to write to the PPU
during "scene transitions" and see if there is anything I can use
to fingerprint scenes...
yeah... i'm not entirely sure of the thought process I had with my
old method... It worked, but it wasn't very clean, or intuitive.
This new method is a lot cleaner, and makes a lot more sense.
Moreover, it enables me to add the 16x16 block sampling
functionality, which is awesome, since animated blocks tend no to
get "banded" or "smeared."
basically, instead of using a referance to a pointer, i'm just
using a double pointer. Aside from looking nice, it also gives my
NES class a more C-like interface, which will come in handy if I
ever want to export ANESE as a libretro core.
I think i've settled on how to expose the inner workings of ANESE.
1) callbacks at key points
2) direct-acess to member variables using _functions.
Anything in ANESE core that starts with an underscore is something
that is _not related to NES emulation_, and is purely there for
instrumentation / debugging.
gui.cc is now _almost_ module agnostic, which is awesome!
The last thing to do is to figure out what's the best way to route
input from the wideNES window to the main window, as the current
solution of _always_ passing input to the emulator is jank af.
I also finally got a ROM loading / unloading solution i'm happy
about. essentially, load/unload have become utility methods on the
shared-state struct.
no more SDL in ANESE core!
Also, with this commit, I can merge back into master, since wideNES
is hidden behind a flag :)
Before I do that though, a bit of cleanup is in order :)
to fix the smearing in MMC3 games, I needed to change which scroll-
values to use depending on if the status bar is at the top / bottom
of the screen. If it's at the top of the screen, then I need to use
the scroll values recorded _after_ the IRQ is fired. If it's at the
bottom of the screen, then I need to use the scroll values recorded
_before_ the IRQ is fired.
Also, turns out with my newer, better sampling methods, I don't
need to use that janky "true 0" filter anymore, so that's gone!
Lastly, I removed all wideNES specific code from the PPU, instead
adding scroll callbacks (so I can register values on my end)
I wanted to use SDL_inprint in multiple renderers, and that meant I
had to modify the library a little bit.
also, I just noticed that the smearing issue is back! Not as bad as
before, but MMC3 games seems to be affected when the screen scrolls
upwards...
wierd.
If a human can figure out which parts of a screen to chop, why
can't wideNES do it automatically? Well, it can! sort of...
I implemented a couple of heuristics that use the state of the PPU
and the Mapper to "guess" what padding values make sense.
For now, I only have 2 implemented:
1) Check if the bgr-mask column is enbled on PPUMASK
- if so, add 8px of padding to the left of the screen
- many games that mask the left also have artifacting on the
right, so mask off 8px on the right too.
2) Check what scanline the Mapper IRQs happened on
- mask-off the rest of the screen
- many games use the Mapper IRQ for status-bars (smb3, kirby)
Now, these are not perfect heuristics by any means, with exceptions
rampant. eg: Lode Runner doesn't have any graphical artifacting on
the right of the screen. eg2: MegaMan IV uses the IRQ for different
reasons, unrelated to status-bars.
Nevertheless, they do a decent job with many games, smb3 and MC Kids
work flawlessly. Moreover, Metroid and Super Mario Bros work great
too, automatically having the entire screen captured (no padding)
While implemeting this feature, I decided to cleanup how I was
reading the NES's state, and implemented a basic Callback system,
enabling arbitrary methods to hook into important parts of the NES
lifecycle. eg: I added callbacks whenever a mapper fires an IRQ,
and a mapper whenever a PPU frame finished drawing (scanline 240)
This had the happy side-effect of fixing my smearing issues, as it
makes a lot more sense to sample the NES's state _before_ the NMI
handler happens, as that state the one that resulted in the current
frame! Just like that, every game I tested worked flawlessly, no
smearing to be found!
Also, this lays a much more solid groundwork for implementing a
proper debugger! It should be easy to take the nastyness that is
ppu_debug.cc and extract it into it's own GUI module! That way,
there will be _no_ state of SDL in ANESE core!
I still haven't figured out the smearing issue, but I _did_ figure
out how to greatly-improve compatibility with many titles!
I added the ability to "pad" the screen, cutting-off any artifacts
at the edge. This means that with a bit of tweaking, games like
SMB3 and MC Kids work near flawlessly!
It _should_ be possible to guess "good" padding values for games by
using some heuristic that relies on current mirroring mode, scroll
direction, and if the PPUMASK is on...
I also updated the README some more, putting a "teaser" for wideNES
on the main page, but splitting off the details into it's own page.
Yeah, it's about time I get wideNES out of the main NES module, it
was starting to make a mess. Instead, wideNES now has it's own
window, with it's own SDL rendering context.
I'd actually like to refactor the emu module some more, possibly
splitting it into sever different ones. eg: emu_core for the main
NES console + frameloop, emu_controls for handling input / movie
support, and take the file i/o and throw that into the menu module.
I'll get around to it eventually :)
Plus, if I can modularize things a bit more, I can merge this
branch into main, and put the entire wideNES module behind a flag!
-----
smearing.
I'm very confused.
I tried a bunch of stuff, and one thing I stumbled on was that if I
use the _last_ frame's scroll values, then I can fix smearing in
some games (smb2, metroid) at the expense of adding it to other
games (smb1)
I'm really not sure why that's happening, but i'll try investigate.
-----
Oh, and I added basic pan/zoom to WideNES, so you can scroll around
the viewport now :D
the last tile-based approach was a very jankyboi, with oversized
tiles that were clipped using SDL, with no memory of a framebuffer.
not great, not great at all.
I revamped the system so that each tile has it's own internal
framebuffer, and updates occur by copying chunks of the main nes
framebuffer into the 4 tiles the screen currently overlaps.
this works really well, and has major upsides like
1) no more black-strips
2) possiblity to save tiles (easy access to framebuffer)
3) easier to work with
I'm still running into some issues though, the biggest of which is
that most games that scroll end up "smearing" the background...
It's hard to explain in text, but if you load up SMB2 or Metroid
for example, whenever you scroll, the stuff you leave behind has
duplicate columns, resulting in a "smearing" effect.
I'm at a bit of a loss as to what might be causing it, as the
scroll-register values being sent to the PPU by those games seem
sensible...
I'll try to fix it somehow lol.
this is really cool!
it totally seems to be working!
Games like smb1 and castlevania work very nicely.
smb2 even (sorta) works!
i'm really excited!
The approach of sniffing the scroll registers is _okay_, but i'm
a bit worried that it will not work properly with games that that
scroll _without_ using the scroll registers.
That said, i'm not going to think about that for now, and cross
that bridge later...
There are some hiccups that need to be ironed out in the current
implementation:
1) sampling the edge of the screen for the background isn't ideal.
- There tend to be artifacts (e.g: smb3) on the edges, so the bg
gets messed up. It might make more sense to sample ~8px into
the framebuffer.
- i'll need to rewrite my sampling implementation from using
SDL clipping, to manually doing manipulations on the
framebuffer
2) inspecting the scroll registers works great, except around 0.
- you'd think that the only times the scroll registers are
written to are when the scroll changes, right? That's not the
case though... many games (eg: smb1) will write the scoll value,
and then later write a couple of 0s. why? good question!
- two problems arise because of this:
a) it's tricky to filter out the dummy 0s from the true 0s
- i'm using a simple heuristic for now, and while it works,
it certaintly isn't robust...
b) games tend to update the scroll register by more than 1
unit at a time
- this is bad since it results in black-stripes of missing
background whenever this occurs.
3) still need to fix going backwards lol
Lastly, here are some things i'd like to do after I have a
somewhat robust system in place:
- Only record the Background layer, avoiding sprites entirely.
- this should get rid of some nasty artifacting when sprites
shuffle off-screen
- Detect when a scene-change occurs, and start a new "plane"
- eg: going through a door in SMB2 should discard the old tiles
and record a fresh set.
- I might be able to detect this in a few ways:
a) checking for CHR memory changes
b) checking for palette changes
c) finding some sort of "average" for a scene (e.g: histogram
of pixel colors throughout the screen)
- this one should work the best I think
- Save tilemaps for future playthroughs
- shouldn't be too hard...
- the tricky bit is saving the tiles w.r.t scenes...
i'm really excited to see how good wideNES turns out, given the
really cool initial results.
I had a thought: wouldn't it be cool to "widescreenify" NES games?
There are a bunch of different ways to approach this, but i'm
experimenting with a fairly general approach: watching the PPU's
scroll registers, and shifting the screen around accordingly...
This is just an initial test, but SMB is already showing some cool
results!
As far as I can tell, i'm the first to experiment with this idea,
which is exciting, because there isn't terribly much new ground to
break in the world of NES emulators!
I'd like to think this commit marks the end of the first phase
of ANESE's development. It's been a long road, but ANESE is
finally at a point where I'm mostly happy with it.
ANESE sure as hell isn't perfect, but hey, it's pretty good!
The core code is pretty clean, the UI code is... acceptable, but
most importantly, ANESE actually plays my favorite NES games!
There is still some work to be done before I'd be comfortable
giving ANESE a v1.0.0 release, but what I have here is still
pretty great.
Let's call this ANESE v0.9.0 :)
-------
So, what next?
Well, contrary to what I said in some earlier commits, I think i'll
continue to work on ANESE a little bit more!
Specifically, i'd like to rewrite the 6502 emulation.
My current implementation is... okay.
It's instruction-level cycle accurate, but I don't think that's
good enough. It really should be sub-instruction level accurate.
Odds are the added accuracy will fix _a lot_ of bugs.
Aside from accuracy though, I have another reason to do a rewrite...
As a Waterloo student, I have to do a Work Term Report on some
technical project i've worked on recently. ANESE is one such project.
Since the report isn't designed to be very long, I'd limit my scope
to just a small aspect of ANESE: the 6502 emulator.
Yes, I could just write the report on how I arrived at my current
implementation, but I think it would be cool to attempt a cleaner
rewrite, and compare and contrast the two versions.
So yeah, stay tuned! I might also post the writeup (once I get
around to it)
I'm pretty happy with how SDL_GUI cleaned up. It's now pretty thin,
with just a bit of nontrivial, ANESE related behavior, namely:
1) Loading a rom passed as an arg
2) The main run loop is not fully generic (queries modules)
Neither of those are that bad though, so i'm calling it for now.
I think that after this, i'll just have 2 more tasks left until I
tag this as ANESE release 1.0.
1) Comment the codebase some more.
It's pretty good as it stands, but I want to give it a once-over.
2) See if I can't just figure out how to use SDL_QueueAudio
Yes, Blargg's Sound_Queue works _fine_, but I'd feel better if I
could say that _all_ of ANESE's sound code is hand-written.
The windows build broke because of a stray #include <Windows.h>
defining some macros that conflict with my enums... I hotfixed it.
The Linux bug is more interesting, and has to do with a compileg
bug present in g++4. It's fixed on later versions though...
I think it's coming along pretty well. In my second pass, I just
need to push more behavior into the modules (that is currently
scattered around the gui.h/cc)
no more janky DEBUG_VARS! Now, there is a defined NES_Params object
that is passed (by const ref) to the NES on construction, which is
passed along to all components that need it, with each component
holding a const ref to the individual fields it cares about.
Plus, I could finally get rid of debug.h/cc, which has outlived
it's usefulness.
What the hell was I thinking, spreading each GUI subsystem across
a bunch of files?
Now, each subsysyem (the NES and the Menu) are organized properly,
with a single file containing their respective init, input
handling, update, output, and clenaup routines.
It's still kinda jank, but it's a lot easier to get things done.
cross-platform support man. it's tricky.
I didn't want to include more libraries, but I really didn't want to
deal with cross-platform pathfinding, or coming up with a custom
config format.
I still had to slap together a janky solution for getting a folder's
absolute path across platforms, but it seems to get the job done...
I fixed up the dmc some more, since I realized it sounded like
death (especially in Kirby's Adventure)
I also refactored and cleaned-up much of the APU. Lookin' good :)
Well, here we are. I implemented my own APU.
It's not great, far from it. Hell, I doubt interupts even remotely
work. I sure as hell won't be passing any APU tests...
But hey, it works in most games!
---
This was my last major task with ANESE.
While there are a _lot_ of things that I could still improve, I
think it might be time for me to ramp down, and start looking at
some other projects to work on. ANESE has taken up a huge chunk of
my time lately...
In the near future, I might push a few more commits to ANESE,
cleaning up some code here, adding some more comments there. Maybe
i'll actually improve the UI a bit.
right, I did a lot here:
1) implemented the DMC. It was a bit tricky, but not as bad as I
had initially thought. I should implement CPU stall better, but w/e
2) I figured out why my pulses sounded like shit! It's because I
accidentally used sweep.val instead of sweep_val, i.e: assigning to
the union on every sweep clock! I fixed that, adn renamed sweep.val
to sweep._val so I don't make that mistake in the future.
3) Why even use `short` samples? It's 2018, we all have FPUs!
---
I also experimented with switching to SDL_QueueAudio instead of
the current Sounds_Queue.h thing I from Blargg, but there seems to
be a gradual desync issue when I use it...
Also, I found that bumping the sample-rate up greatly improved the
audio-quality, espeically on pulses. iirc, it's because I am doing
sampling in a shitty way, when I should be sampling more around key
inflection points.
See http://www.slack.net/~ant/bl-synth/ for what I mean...
(that's how Blargg's nes_snd_emu does it, and it sounds awesome)
as someone who hasn't taken a signals processing course, figuring
out how to get these things working was a real pain.
Luckilly, Wikipedia is aweomse, and straight-up provides an
algorithm for both lopass and hipass filters. All it needed was a
bit of modification, and bam, easy as that!
----
As a sidenote, as i've been developing the APU, i've compared it to
some of the other APUs out there (including Blarrg's), and it
definately sounds shittier.
Maybe it's because i don't have DMC implemented yet? Unlikely...
Odds are i'll have to look into actual waveform synthesis if I want
it to sound better, which will be fun-on-the-bun.
I had the length-counter conditions messed up, which made things
sound pretty bad. After correcting that, the audio started to sound
a lot better.
Pulse1 and Pulse2 were relatively straightforward to implement,
although it's a bit tricky understanding the wording on the wiki.
Just Triangle and DMC to go!
(oh, and I should probably implement those high-pass and low-pass
filters too)
Would you look at that! It actually wasn't _that_ bad once I read
through the docs some more. Turns out that I don't _really_ need to
deal with complicated waveform sysnthesis, since the nesdev wiki
gives a handy-dandy lookup table for what the NES should output
given the volumes of all the channels!
I'd imagine getting pulse1, pulse2, and triangle working shouldn't
be that hard. DMC might be trickier though...
Oh, and I still need to hammer on APU IRQ timings.
I have no idea what i'm doing lol.
I'm starting with the noise channel, since I think that might be the
easiest to do?
I _think_ I implemented it properly, since sniffing the output gives
reasonable volumes, but I don't really understand how to generate
sound waves from it yet...
