summaryrefslogtreecommitdiffstatshomepage
path: root/src/lib/util/hashing.h
diff options
context:
space:
mode:
author Aaron Giles <aaron@aarongiles.com>2012-02-16 09:47:18 +0000
committer Aaron Giles <aaron@aarongiles.com>2012-02-16 09:47:18 +0000
commitf0823886a66100e193d6eeb0402eb872a67fa07d (patch)
treea68b35942d63e5fcaf2311812dba976ad18cd5b6 /src/lib/util/hashing.h
parente6dad3759374373e3ce69a76869f16e83ba74df5 (diff)
Major CHD/chdman update. The CHD version number has been increased
from 4 to 5. This means any diff CHDs will no longer work. If you absolutely need to keep the data for any existing ones you have, find both the diff CHD and the original CHD for the game in question and upgrade using these commands: rename diff\game.dif diff\game-old.dif chdman copy -i diff\game-old.dif -ip roms\game.chd -o diff\game.dif -op roms\game.chd -c none Specifics regarding this change: Defined a new CHD version 5. New features/behaviors of this version: - support for up to 4 codecs; each block can use 1 of the 4 - new LZMA codec, which tends to do better than zlib overall - new FLAC codec, primarily used for CDs (but can be applied anywhere) - upgraded AVHuff codec now uses FLAC for encoding audio - new Huffman codec, used to catch more nearly-uncompressable blocks - compressed CHDs now use a compressed map for significant savings - CHDs now are aware of a "unit" size; each hunk holds 1 or more units (in general units map to sectors for hard disks/CDs) - diff'ing against a parent now diffs at the unit level, greatly improving compression Rewrote and modernized chd.c. CHD versions prior to 3 are unsupported, and version 3/4 CHDs are only supported for reading. Creating a new CHD now leaves the file open. Added methods to read and write at the unit and byte level, removing the need to handle this manually. Added metadata access methods that pass astrings and dynamic_buffers to simplify the interfaces. A companion class chd_compressor now implements full multithreaded compression, analyzing and compressing multiple hunks independently in parallel. Split the codec implementations out into a separate file chdcodec.* Updated harddisk.c and cdrom.c to rely on the caching/byte-level read/ write capabilities of the chd_file class. cdrom.c (and chdman) now also pad CDs to 4-frame boundaries instead of hunk boundaries, ensuring that the same SHA1 hashes are produced regardless of the hunk size. Rewrote chdman.exe entirely, switching from positional parameters to proper options. Use "chdman help" to get a list of commands, and "chdman help <command>" to get help for any particular command. Many redundant commands were removed now that additional flexibility is available. Some basic mappings: Old: chdman -createblankhd <out.chd> <cyls> <heads> <secs> New: chdman createhd -o <out.chd> -chs <cyls>,<heads>,<secs> Old: chdman -createuncomphd <in.raw> <out.chd> .... New: chdman createhd -i <in.raw> -o <out.chd> -c none .... Old: chdman -verifyfix <in.chd> New: chdman verify -i <in.chd> -f Old: chdman -merge <parent.chd> <diff.chd> <out.chd> New: chdman copy -i <diff.chd> -ip <parent.chd> -o <out.chd> Old: chdman -diff <parent.chd> <compare.chd> <diff.chd> New: chdman copy -i <compare.chd> -o <diff.chd> -op <parent.chd> Old: chdman -update <in.chd> <out.chd> New: chdman copy -i <in.chd> -o <out.chd> Added new core file coretmpl.h to hold core template classes. For now just one class, dynamic_array<> is defined, which acts like an array of a given object but which can be appended to and/or resized. Also defines dynamic_buffer as dynamic_array<UINT8> for holding an arbitrary buffer of bytes. Expect to see these used a lot. Added new core helper hashing.c/.h which defines classes for each of the common hashing methods and creator classes to wrap the computation of these hashes. A future work item is to reimplement the core emulator hashing code using these. Split bit buffer helpers out into C++ classes and into their own public header in bitstream.h. Updated huffman.c/.h to C++, and changed the interface to make it more flexible to use in nonstandard ways. Also added huffman compression of the static tree for slightly better compression rates. Created flac.c/.h as simplified C++ wrappers around the FLAC interface. A future work item is to convert the samples sound device to a modern device and leverage this for reading FLAC files. Renamed avcomp.* to avhuff.*, updated to C++, and added support for FLAC as the audio encoding mechanism. The old huffman audio is still supported for decode only. Added a variant of core_fload that loads to a dynamic_buffer. Tweaked winwork.c a bit to not limit the maximum number of processors unless the work queue was created with the WORK_QUEUE_FLAG_HIGH_FREQ option. Further adjustments here are likely going to be necessary. Fixed bug in aviio.c which caused errors when reading some AVI files.
Diffstat (limited to 'src/lib/util/hashing.h')
-rw-r--r--src/lib/util/hashing.h245
1 files changed, 245 insertions, 0 deletions
diff --git a/src/lib/util/hashing.h b/src/lib/util/hashing.h
new file mode 100644
index 00000000000..8bb59b9c5cc
--- /dev/null
+++ b/src/lib/util/hashing.h
@@ -0,0 +1,245 @@
+/***************************************************************************
+
+ hashing.h
+
+ Hashing helper classes.
+
+****************************************************************************
+
+ Copyright Aaron Giles
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions are
+ met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name 'MAME' nor the names of its contributors may be
+ used to endorse or promote products derived from this software
+ without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY AARON GILES ''AS IS'' AND ANY EXPRESS OR
+ IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ DISCLAIMED. IN NO EVENT SHALL AARON GILES BE LIABLE FOR ANY DIRECT,
+ INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ POSSIBILITY OF SUCH DAMAGE.
+
+***************************************************************************/
+
+#pragma once
+
+#ifndef __HASHING_H__
+#define __HASHING_H__
+
+#include "osdcore.h"
+#include "astring.h"
+#include "zlib.h"
+#include "md5.h"
+#include "sha1.h"
+
+
+//**************************************************************************
+// TYPE DEFINITIONS
+//**************************************************************************
+
+
+// ======================> SHA-1
+
+// final digest
+struct sha1_t
+{
+ bool operator==(const sha1_t &rhs) const { return memcmp(m_raw, rhs.m_raw, sizeof(m_raw)) == 0; }
+ bool operator!=(const sha1_t &rhs) const { return memcmp(m_raw, rhs.m_raw, sizeof(m_raw)) != 0; }
+ operator UINT8 *() { return m_raw; }
+ const char *as_string(astring &buffer);
+ bool from_string(const char *string);
+ UINT8 m_raw[20];
+ static const sha1_t null;
+};
+
+// creation helper
+class sha1_creator
+{
+public:
+ // construction/destruction
+ sha1_creator() { reset(); }
+
+ // reset
+ void reset() { sha1_init(&m_context); }
+
+ // append data
+ void append(const void *data, UINT32 length) { sha1_update(&m_context, length, reinterpret_cast<const UINT8 *>(data)); }
+
+ // finalize and compute the final digest
+ sha1_t finish()
+ {
+ sha1_t result;
+ sha1_final(&m_context);
+ sha1_digest(&m_context, sizeof(result.m_raw), result.m_raw);
+ return result;
+ }
+
+ // static wrapper to just get the digest from a block
+ static sha1_t simple(const void *data, UINT32 length)
+ {
+ sha1_creator creator;
+ creator.append(data, length);
+ return creator.finish();
+ }
+
+protected:
+ // internal state
+ struct sha1_ctx m_context; // internal context
+};
+
+
+
+// ======================> MD5
+
+// final digest
+struct md5_t
+{
+ bool operator==(const md5_t &rhs) const { return memcmp(m_raw, rhs.m_raw, sizeof(m_raw)) == 0; }
+ bool operator!=(const md5_t &rhs) const { return memcmp(m_raw, rhs.m_raw, sizeof(m_raw)) != 0; }
+ operator UINT8 *() { return m_raw; }
+ const char *as_string(astring &buffer);
+ bool from_string(const char *string);
+ UINT8 m_raw[16];
+ static const md5_t null;
+};
+
+// creation helper
+class md5_creator
+{
+public:
+ // construction/destruction
+ md5_creator() { reset(); }
+
+ // reset
+ void reset() { MD5Init(&m_context); }
+
+ // append data
+ void append(const void *data, UINT32 length) { MD5Update(&m_context, reinterpret_cast<const unsigned char *>(data), length); }
+
+ // finalize and compute the final digest
+ md5_t finish()
+ {
+ md5_t result;
+ MD5Final(result.m_raw, &m_context);
+ return result;
+ }
+
+ // static wrapper to just get the digest from a block
+ static md5_t simple(const void *data, UINT32 length)
+ {
+ md5_creator creator;
+ creator.append(data, length);
+ return creator.finish();
+ }
+
+protected:
+ // internal state
+ struct MD5Context m_context; // internal context
+};
+
+
+
+// ======================> CRC-32
+
+// final digest
+struct crc32_t
+{
+ bool operator==(const crc32_t &rhs) const { return m_raw == rhs.m_raw; }
+ operator UINT32() const { return m_raw; }
+ const char *as_string(astring &buffer);
+ bool from_string(const char *string);
+ UINT32 m_raw;
+ static const crc32_t null;
+};
+
+// creation helper
+class crc32_creator
+{
+public:
+ // construction/destruction
+ crc32_creator() { reset(); }
+
+ // reset
+ void reset() { m_accum.m_raw = 0; }
+
+ // append data
+ void append(const void *data, UINT32 length) { m_accum.m_raw = crc32(m_accum, reinterpret_cast<const Bytef *>(data), length); }
+
+ // finalize and compute the final digest
+ crc32_t finish() { return m_accum; }
+
+ // static wrapper to just get the digest from a block
+ static crc32_t simple(const void *data, UINT32 length)
+ {
+ crc32_creator creator;
+ creator.append(data, length);
+ return creator.finish();
+ }
+
+protected:
+ // internal state
+ crc32_t m_accum; // internal accumulator
+};
+
+
+
+// ======================> CRC-16
+
+// final digest
+struct crc16_t
+{
+ bool operator==(const crc16_t &rhs) const { return m_raw == rhs.m_raw; }
+ operator UINT16() const { return m_raw; }
+ const char *as_string(astring &buffer);
+ bool from_string(const char *string);
+ UINT16 m_raw;
+ static const crc16_t null;
+};
+
+// creation helper
+class crc16_creator
+{
+public:
+ // construction/destruction
+ crc16_creator() { reset(); }
+
+ // reset
+ void reset() { m_accum.m_raw = 0xffff; }
+
+ // append data
+ void append(const void *data, UINT32 length);
+
+ // finalize and compute the final digest
+ crc16_t finish() { return m_accum; }
+
+ // static wrapper to just get the digest from a block
+ static crc16_t simple(const void *data, UINT32 length)
+ {
+ crc16_creator creator;
+ creator.append(data, length);
+ return creator.finish();
+ }
+
+protected:
+ // internal state
+ crc16_t m_accum; // internal accumulator
+};
+
+
+#endif // __HASHING_H__