bsnes/snesreader/unrar/arcread.cpp
byuu 989648c21c Update to bsnes v062 release.
Major accuracy improvements have happened over the past few days. They easily warrant a new beta release.
First, it turns out that every emulator to date; not only for the SNES, but for the Apple II GS as well, incorrectly computed ADC (add) and SBC (subtract) flags in BCD (binary-coded decimal) mode. At least fifteen years of emulating the 65816 processor, at least five known investigations into their behavior, and we all still had it wrong.
So I wrote some tests that dumped every possible combination of adc and sbc with every possible input and every possible flag, and recorded both the accumulator result and status flag register. From here, blargg figured out the underlying trick: the CPU was computing overflow before the top-nibble's BCD correction pass. With the routines rewritten, bsnes perfectly matches real hardware now.
Next, some background. The whole reason I got into SNES emulation was because I was tired of writing code that ran perfectly fine on emulators, but failed miserably on real hardware. The number one problem was emulators allowing video RAM to be written while the screen was being rendered. This single bug has broken dozens of fan translations and ROM hacks. Some have been updated to work around this bug, and many others are left in a permanently broken state (such as the translations of Dragon Quest I & II and Sailor Moon: Another Story, to name just two.) After asking emulator authors to fix this since 1997, I finally had enough in 2004 and started on bsnes. For this particular bug, I'm very happy to report that all but one SNES emulator now properly blocks these invalid accesses. Although sadly one still offers a configuration setting for backwards compatibility with these translations. What an ironic shame ... emulating an emulator. And in the process, sapping the motivation to ever go back and fix these 
titles to ever run on real hardware. But I digress ...
The second biggest problem that causes software created under emulation to break on real hardware has, without a doubt, been the hardware delays as the S-CPU computes MUL (multiplication) and DIV (division) results. To date, whenever you used this hardware functionality, emulators have immmediately furnished the correct results. But on real hardware, multiplication requires eight CPU cycles, and division requires sixteen. Each step computes one bit of the source operand and updates the results. Reading the output registers early thus provides the partially computed results.
This is obscure. It isn't well known, and many people writing software for the SNES probably aren't even aware of this limitation. Because of the partial computation results, outright delaying the computation would break many commercial software titles. But by not emulating the delay at all, we were causing a great disservice to anyone wishing to use an emulator for development purposes.
Now, once again, thanks to blargg's algorithm help, he has determined the underlying multiplication and division algorithms. Combined with my expertise of SNES analysis and hardware testing, I was able to determine when and how the ALU (arithmetic logic unit) stepped through each work cycle. Our work combined, bsnes now also perfectly emulates the hardware MUL and DIV delays.
Again, this isn't going to fix commercial software titles. They would have realized that they were reading back invalid MUL and DIV values, and fixed their code. This is for all of the software developed using emulators. This is an extension of my commitment to create a hardware emulator, and not a video game emulator.
We also verified that the S-PPU multiplication interface does indeed return instant results with no delay. So emulation of that interface was already correct.
I'm only labelling this release a beta because it hasn't been tested. But I'm fairly confident that it is stable, and I seriously recommend upgrading from v060 or prior releases. This is easily one of the last major pieces missing from emulation.
The last notable elements are: S-CPU auto joypad poll timing, S-CPUr1 HDMA crash detection, S-CPU<>S-SMP port ORing, S-SMP timer glitching, S-DSP mute pulse, and full cycle-level emulation of the S-PPU. With all of the aforementioned items, I will consider a v1.0 release of bsnes ;)
Lastly, I'll post this screenshot just for fun. When d4s translated Breath of Fire II to German, he added some code that relies on the incorrect emulation of the DIV register to detect emulators. With this emulated properly, you now see the following screen:
./sshots/bs_349.png
Sorry to spoil that, but the secret's already out, as the MESS team reported on it publicly already.
I intend to add pseudo-randomness support shortly, which should eliminate one of the last vectors possible to distinguish bsnes from real hardware :)
A million thanks to blargg for making this release possible.
2010-03-13 23:48:54 +00:00

314 lines
7.5 KiB
C++

#include "rar.hpp"
#include "unrar.h"
#include "unicode.hpp"
#include "encname.hpp"
// arcread.cpp
unrar_err_t Archive::ReadHeader()
{
CurBlockPos=Tell();
#ifndef SFX_MODULE
if (OldFormat)
{
ReadOldHeader();
if ( Raw.Size() == 0 )
return unrar_err_arc_eof; // right at end of file
if ( Raw.PaddedSize() > 0 ) // added check
return unrar_err_corrupt; // missing data
return unrar_ok;
}
#endif
Raw.Reset();
// (removed decryption)
Raw.Read(SIZEOF_SHORTBLOCKHEAD);
if (Raw.Size()==0)
{
return unrar_err_arc_eof; // right at end of file
}
Raw.Get(ShortBlock.HeadCRC);
byte HeadType;
Raw.Get(HeadType);
ShortBlock.HeadType=(HEADER_TYPE)HeadType;
Raw.Get(ShortBlock.Flags);
Raw.Get(ShortBlock.HeadSize);
if (ShortBlock.HeadSize<SIZEOF_SHORTBLOCKHEAD)
{
return unrar_err_corrupt; // invalid HeadSize
}
// catches unknown block types and reading something that isn't even a header
check( MARK_HEAD <= ShortBlock.HeadType && ShortBlock.HeadType <= ENDARC_HEAD );
if (ShortBlock.HeadType==COMM_HEAD)
Raw.Read(SIZEOF_COMMHEAD-SIZEOF_SHORTBLOCKHEAD);
else
if (ShortBlock.HeadType==MAIN_HEAD && (ShortBlock.Flags & MHD_COMMENT)!=0)
Raw.Read(SIZEOF_NEWMHD-SIZEOF_SHORTBLOCKHEAD);
else
Raw.Read(ShortBlock.HeadSize-SIZEOF_SHORTBLOCKHEAD);
if ( Raw.PaddedSize() > 0 ) // fewer than requested bytes read above?
return unrar_err_corrupt; // missing data
NextBlockPos=CurBlockPos+ShortBlock.HeadSize;
switch(ShortBlock.HeadType)
{
case MAIN_HEAD:
*(BaseBlock *)&NewMhd=ShortBlock;
Raw.Get(NewMhd.HighPosAV);
Raw.Get(NewMhd.PosAV);
check( Raw.ReadPos == Raw.DataSize ); // we should have read all fields
break;
case FILE_HEAD:
case NEWSUB_HEAD:
{
FileHeader *hd=ShortBlock.HeadType==FILE_HEAD ? &NewLhd:&SubHead;
*(BaseBlock *)hd=ShortBlock;
Raw.Get(hd->PackSize);
Raw.Get(hd->UnpSize);
Raw.Get(hd->HostOS);
Raw.Get(hd->FileCRC);
Raw.Get(hd->FileTime);
Raw.Get(hd->UnpVer);
Raw.Get(hd->Method);
Raw.Get(hd->NameSize);
Raw.Get(hd->FileAttr);
if (hd->Flags & LHD_LARGE)
{
Raw.Get(hd->HighPackSize);
Raw.Get(hd->HighUnpSize);
}
else
{
hd->HighPackSize=hd->HighUnpSize=0;
if (hd->UnpSize==0xffffffff)
{
// TODO: what the heck is this for anyway?
hd->UnpSize=0;
hd->HighUnpSize=0x7fffffff;
}
}
hd->FullPackSize=int32to64(hd->HighPackSize,hd->PackSize);
hd->FullUnpSize=int32to64(hd->HighUnpSize,hd->UnpSize);
if ( int32to64( 1, 0 ) == 0 && (hd->HighPackSize || hd->HighUnpSize) )
return unrar_err_huge;
char (&FileName) [sizeof hd->FileName] = hd->FileName; // eliminated local buffer
int NameSize=Min(hd->NameSize,sizeof(FileName)-1);
Raw.Get((byte *)FileName,NameSize);
FileName[NameSize]=0;
if (hd->HeadType==NEWSUB_HEAD)
{
// have to adjust this, even through we're ignoring this block
NextBlockPos+=hd->FullPackSize;
break;
}
else
if (hd->HeadType==FILE_HEAD)
{
if (hd->Flags & LHD_UNICODE)
{
EncodeFileName NameCoder;
int Length=strlen(FileName);
if (Length==hd->NameSize)
{
UtfToWide(FileName,hd->FileNameW,sizeof(hd->FileNameW)/sizeof(hd->FileNameW[0])-1);
WideToChar(hd->FileNameW,hd->FileName,sizeof(hd->FileName)/sizeof(hd->FileName[0])-1);
ExtToInt(hd->FileName,hd->FileName);
}
else
{
Length++;
NameCoder.Decode(FileName,(byte *)FileName+Length,
hd->NameSize-Length,hd->FileNameW,
sizeof(hd->FileNameW)/sizeof(hd->FileNameW[0]));
}
if (*hd->FileNameW==0)
hd->Flags &= ~LHD_UNICODE;
}
else
*hd->FileNameW=0;
ConvertUnknownHeader();
}
if (hd->Flags & LHD_SALT)
Raw.Get(hd->Salt,SALT_SIZE);
hd->mtime.SetDos(hd->FileTime);
if (hd->Flags & LHD_EXTTIME)
{
ushort Flags;
Raw.Get(Flags);
// Ignore additional time information
for (int I=0;I<4;I++)
{
uint rmode=Flags>>(3-I)*4;
if ((rmode & 8)==0)
continue;
if (I!=0)
{
uint DosTime;
Raw.Get(DosTime);
}
// skip time info
int count=rmode&3;
for (int J=0;J<count;J++)
{
byte CurByte;
Raw.Get(CurByte);
}
}
}
NextBlockPos+=hd->FullPackSize;
bool CRCProcessedOnly=(hd->Flags & LHD_COMMENT)!=0;
HeaderCRC=~Raw.GetCRC(CRCProcessedOnly)&0xffff;
if (hd->HeadCRC!=HeaderCRC)
return unrar_err_corrupt;
check( CRCProcessedOnly == false ); // I need to test on archives where this doesn't hold
check( Raw.ReadPos == Raw.DataSize ); // we should have read all fields
}
break;
#ifndef SFX_MODULE
// Handle these block types just so we can adjust NextBlockPos properly
case PROTECT_HEAD:
Raw.Get(ProtectHead.DataSize);
NextBlockPos+=ProtectHead.DataSize;
break;
case SUB_HEAD:
Raw.Get(SubBlockHead.DataSize);
NextBlockPos+=SubBlockHead.DataSize;
break;
#endif
default:
if (ShortBlock.Flags & LONG_BLOCK)
{
uint DataSize;
Raw.Get(DataSize);
NextBlockPos+=DataSize;
}
break;
}
HeaderCRC=~Raw.GetCRC(false)&0xffff;
CurHeaderType=ShortBlock.HeadType;
// (removed decryption)
if (NextBlockPos<CurBlockPos+Raw.Size())
return unrar_err_corrupt; // next block isn't past end of current block's header
// If pos went negative, then unrar_pos_t is only 32 bits and it overflowed
if ( NextBlockPos < 0 )
return unrar_err_huge;
return unrar_ok;
}
// Rar.Read()s are checked by caller of ReadOldHeader() (see above)
#ifndef SFX_MODULE
int Archive::ReadOldHeader()
{
Raw.Reset();
if (CurBlockPos<=SFXSize)
{
Raw.Read(SIZEOF_OLDMHD);
Raw.Get(OldMhd.Mark,4);
Raw.Get(OldMhd.HeadSize);
Raw.Get(OldMhd.Flags);
NextBlockPos=CurBlockPos+OldMhd.HeadSize;
CurHeaderType=MAIN_HEAD;
}
else
{
OldFileHeader OldLhd;
Raw.Read(SIZEOF_OLDLHD);
NewLhd.HeadType=FILE_HEAD;
Raw.Get(NewLhd.PackSize);
Raw.Get(NewLhd.UnpSize);
Raw.Get(OldLhd.FileCRC);
Raw.Get(NewLhd.HeadSize);
Raw.Get(NewLhd.FileTime);
Raw.Get(OldLhd.FileAttr);
Raw.Get(OldLhd.Flags);
Raw.Get(OldLhd.UnpVer);
Raw.Get(OldLhd.NameSize);
Raw.Get(OldLhd.Method);
NewLhd.Flags=OldLhd.Flags|LONG_BLOCK;
NewLhd.UnpVer=(OldLhd.UnpVer==2) ? 13 : 10;
NewLhd.Method=OldLhd.Method+0x30;
NewLhd.NameSize=OldLhd.NameSize;
NewLhd.FileAttr=OldLhd.FileAttr;
NewLhd.FileCRC=OldLhd.FileCRC;
NewLhd.FullPackSize=NewLhd.PackSize;
NewLhd.FullUnpSize=NewLhd.UnpSize;
NewLhd.mtime.SetDos(NewLhd.FileTime);
// (removed other times)
Raw.Read(OldLhd.NameSize);
Raw.Get((byte *)NewLhd.FileName,OldLhd.NameSize);
NewLhd.FileName[OldLhd.NameSize]=0;
// (removed name case conversion)
*NewLhd.FileNameW=0;
if (Raw.Size()!=0)
NextBlockPos=CurBlockPos+NewLhd.HeadSize+NewLhd.PackSize;
CurHeaderType=FILE_HEAD;
}
return(NextBlockPos>CurBlockPos ? Raw.Size():0);
}
#endif
// (removed name case and attribute conversion)
bool Archive::IsArcDir()
{
return((NewLhd.Flags & LHD_WINDOWMASK)==LHD_DIRECTORY);
}
bool Archive::IsArcLabel()
{
return(NewLhd.HostOS<=HOST_WIN32 && (NewLhd.FileAttr & 8));
}
// TODO: use '\\' on Windows?
char const CPATHDIVIDER = '/';
#define charnext(s) ((s)+1)
void Archive::ConvertUnknownHeader()
{
if (NewLhd.UnpVer<20 && (NewLhd.FileAttr & 0x10))
NewLhd.Flags|=LHD_DIRECTORY;
if (NewLhd.HostOS>=HOST_MAX)
{
if ((NewLhd.Flags & LHD_WINDOWMASK)==LHD_DIRECTORY)
NewLhd.FileAttr=0x10;
else
NewLhd.FileAttr=0x20;
}
{
for (char *s=NewLhd.FileName;*s!=0;s=charnext(s))
{
if (*s=='/' || *s=='\\')
*s=CPATHDIVIDER;
}
}
// (removed Apple Unicode handling)
for (wchar *s=NewLhd.FileNameW;*s!=0;s++)
{
if (*s=='/' || *s=='\\')
*s=CPATHDIVIDER;
}
}