plain text version (hold shift to open new window)

homepage  The Paulski Pages

TinyHexer and BartPE

Quick Intro: A Live CD (one that runs an operating system) and one capable of running Windows programs is a real boon. A BartPE CD is just such a bootable CD. We hope to demonstrate that it is not that hard to create one. In doing so we also want to show how to add programs to the CD. We have chosen TinyHexer as our first example because it can be a very useful and powerful tool and is the only disk editor that we have come across that can run from a GUI (graphical user interface) on a boot CD. A list of pre-compiled plug-ins is included in an appendix at the bottom of this page.

We also hope to be able to help demystify the rather esoteric nature of hex-editing by outlining some of its basics along with a few examples using TinyHexer. BartPE Live CDs are especially useful for all sorts of troubleshooting and recovery purposes, notably when one is unable to normally access a hard drive's file system. As with WinXP, basic USB2 support is included. Autodetection of inserted USB drives can be problematic but we generally have had no difficulty accessing them if they are attached to the PC prior to boot-up.

NB: If your hard drives contain DDO (also known as Software Translation) and particularly if you are using RAID or SCSI or NT Dynamic Disks then please read the specific cautions1 in the footnotes.  (The importance of appropriate back-up can never be over-stressed).

Outline of these two applications

TinyHexer is a freeware Disk Hex-Editor that runs from a Windows Interface. It is supported under Windows 2000/XP/PE/2003 and, as long as it is run with admin status2, has functioned, without problems, in our hands under Vista. A BartPE Live CD has become the paradigm for a Windows operating system running from a CD. It is a de facto, though cut down version, of Windows XP. This pairing is the best (in fact the only) combination that we have found for directly editing hard drives from a Windows graphical interface running from a Live CD. Nor do we know of any equivalent graphical interface that will run from a Live Linux CD , though some command line utilities do exist. For those that need to do such editing this is a real marriage of convenience. There is a legal pre-requisite that you have a Windows XP or 2003 installation CD (and its licence) in order to create a BartPE CD. Unlike its hard drive mother operating system, no Windows activation is however required and the CD will function on a wide range of hardware - your licence agreement allowing this of course.

BartPE Plug-Ins

Plug-Ins allow additional programs, to those already preconfigured (see Appendix), to be added to a Bart PE CD. These additions are mostly Preinstalled into the Environment from a specially prepared or provided .cab file that contains all the necessary files for the supported application. When the [Add] button is used during a build, one just points it to where the .cab file has been stored and it should then simply be unpacked into the Bart plugins folder. Some provide unpacked plug-ins which you must manually copy into the Bart plugins folder; for example the one for TBU's CopyWipe program. Each unpacked plugin should contain one .htm file with instructions on how it should be used.

A number of default plug-ins will have already been setup when a BartPE .iso image file is compiled. Many may be downloaded from their respective websites. The TinyHexer plug-in is no longer available on the mirkes.de website but can be downloaded from Megaupload. There is a nag wait if you are not a member. We suggest you keep any such new third party plug-in .cab files in a bartcabs (or whatever else you call it) folder somewhere handy, which should make it easy to find them later-on when customising the PE build. [For those needing recovery software we would particularly point out that there are a number of very useful plug-in .cab files available at the bottom of the runtime.org download page]. We ourselves have compiled a pqutils.cab plugin for grabbing HDD information outlined at Read/Edit Hard Drive Details using PartitionInfo and PTEdit.

The TinyHexer .cab file contains the small version of the application which runs quite OK in most of the NT-based OSes (including a PE CD). There is also a medium version as well as a full version that will give one more options when used under a normal Windows installation. We recommend the full version (or other hex editors3) for doing this but do note that, even though they are still available from the site, the creator, as stated in his blog, has stopped developing software.

Installing the BartPE Builder

First of all have your Windows XP (or Windows 2003)  CD or installation files and any BartPE .cab files you intend to compile at the ready. There are a few different approaches but we recommend you download and run the self-installing pebuilder3110a.exe from a mirror of your choice. If you don't have your plugins or Windows installation files at the ready just exit the intitial install and restart the PE Builder when ready.

Compile BartPE #1 Fig 1.

When you have entered the source to the Windows Installation files (say D: for a CD in the D: drive) you can hit the Plugins button. [Hint; if you want to prevent a CD from auto-running when it is inserted hold down the Shift Key while the CD is inserted and has had time to be recognised].

BartPE build #2 Fig 2.

Basic Introduction to 'Hex'

When you fire-up TinyHexer (from any environment) and open a disk or a partition or a file, an internal window should be opened/loaded and have three main sections within it. More than one item (with an internal window) may be open simultaneously but try to have just one open at a time to keep things simple, while learning how to use the application.

One can select an area of code by highlighting it in either the hex or the text area. Its matching areas get highlighted at the same time. This text will usually be completely garbled when it represents binary code (such as assembly code or bitmap images) but readable as human text when it represents textual programming or text-based files. Editing can be done either by entering/changing hex values in the central area or by entering/changing text in the right hand area.

One potentially very confusing area about any "block representation" of data (as in disks and drives and RAM) is to do with the way the offsets are referenced. It is normal for the first byte or the first sector of a range to have the value of 0 and not that of 1. Thus the first sector of a hard drive is sector 0 (0x00); (it is common to write a double zero to indicate both of a byte's nybbles and this can become very important when "endian-ness" comes into play). The first 63 (or 0x3F) sectors, which make up track 0, are sectors 0 through 62 and not 1 through 63. The last LBA sector address of a disk with 3,000,000 sectors is 2,999,999. The last byte's offset of sector 0 is similarly 0x1FF (511) and not 0x200 (512). And so on and so on. This all may take a bit of getting used-to but can be critical in understanding the geometry, when moving around block data and particularly when calculating and editing actual data. Another way that can help to fix this concept in the mind is to note that when we are 1 year old we are in our 2nd year or, if you prefer, we are "0 years old" when in our 1st year.

Basic Disk Hex Fig 3.

Another area that is ready made for confusion is 'endian-ness'. A hex editor's display shows the bytes in the same physical sequence that they are represented on disk. However when more than one byte  is read by the system (and even more confusingly when less than a whole byte is included, as in FAT12 allocation tables and with partition table CHS values) the bytes will actually be read in reverse order in little endian systems such as used by a PC's CPU. Whole single bytes are usually straighforward enough but two bytes (a hex word) or four bytes (a hex d-word) become manifestly different. As an example, the sector signature (the last two bytes of a boot sector) are 0x55 and 0xAA respectively. If these are read as a word and not as two separate bytes then the mathematical value becomes 0xAA55 (43605) and not 0x55AA (21930) if little endian. What is perhaps instructive and a bit of fun is to convert the 0xAA55 value to binary when it becomes 1010101001010101; is it a co-incidental pallindrome? Examples of big endian systems are a motorola CPU and the TCP/IP packets in networking. Don't let the specifics worry you for the moment - we only include it for the purpose of more general understanding. It can, for example, explain why some Windows registry values are not what you might expect them to be.

Getting Started with TinyHexer/BartPE

We will assume you have created a basic BartPE CD with just the TinyHexer plug-in added to it. When one boots to the CD it is initially a bit like booting a normal Windows XP system though possibly a bit faster or a bit slower than usual. Suggest you simply ignore any prompts for Network Support and then from GO >> Programs >> mirkes.de Tiny Hex Editor (Fig 4). There will be no Windows Explorer/My Computer but the A43 File Management Utility should be capable of navigating your file system and so forth as the need arises.

Bart Go Menu Fig 4.

Since we are, initially at least, going to address Disk Editing  you could go to the TinyHexer "File" menu and choose "Disk" and then "Open Drive". You should get a dialog box like that in Fig 5.

Tiny Hex Drives Fig 5.

Files, Drives and Partitions can all be opened into a display window (as in Fig 3) from the main menubar's "File" menu. Files are opened as normal from its "Open" sub-menu and Drives/Partitions from its "Disk" sub-menu. If the inset window is maximized then its description appears in the main title bar - otherwise each window has the description in its own title bar. The bottom left of the status bar in the application's main window always shows two values. The size (in hex) of the active window's loaded data and next to it the absolute position (also in hex) of the cursor. Files are opened in their entirity but block devices (by default) with just one sector at a time. If you want to open a larger block of data from a drive, say the whole of Track 0, then you need to specify that the correct number of sectors are loaded. For a full track that would be 63 (0x3F) sectors.

You can also create a brand new file and enter either hex or text into it before saving it with a name (file extension optional) of your choice. When block devices (essentially all drives) have their hex edited the changes show-up straightaway with different coloured text but are not commited to disk until you demand this or agree to save the changes when prompted. Just remember there is no undo after you have agreed to write the changes. Edited text, with the changed colouration, can however be undone prior to writing/saving the definitive changes. A blank floppy diskette is a good disk to load-up and to then experiment with in order to gain some familiarity with the software, without endangering your main system.

Text files opened and edited in text editors such as notepad should never be particularly problematic but binary files opened in such applications can be because they can unmask hidden EOF (end of file) and other markers. Such binary files are best edited using a hex editor because "what you will see is what you will get". Seeing "what you will get" is one of the main reasons that we like using a graphical and not a command line based hex editor; you can see the changes before you commit to them.

Example 1: Back-up the MBR to a file.

In order to access the MBR you must (as in Fig 5) choose a PHYSICAL DRIVE and not a partition. A physical drive will begin with Sector 0 of the whole hard drive but if you chose a partition it would start with the first sector of the partition, which  is a PBS (partition boot sector) and not an MBR (master boot record). You can check the Readonly box if you want to ensure that you cannot directly write chunks of data to the disk. You should however still be able to access any accessible partitions via their file systems. Even if the Readonly box is unchecked you should get prompted once again before any changes do actually get written to the disk - so we always leave it unchecked. You must leave it unchecked if you want to make a copy of the sector by writing the copy directly to another part of a drive (or whenever you want to edit any disk sectors directly).

Let us say you wanted to make a back-up of the MBR. Ensure you leave "Load 0x01 sectors at a time" and the "First Sector 0" values alone. Then hit OK to open the editing window. If you had checked ReadOnly you must next "Select All" from the Edit Menu and then "Save Selection" from the File Menu. You can also do this with Readonly unchecked. If Readonly is unchecked you can alternatively simply select  "Save As" from the File Menu to save the whole of the already loaded first sector to a file somewhere of your choosing. Use the drop down menu at the top of the Save dialog box to navigate to where you want the file stored and give it any meaningful name of your choosing. The resultant file (say called mbrdisk1.bin or mbrmaxtor.bin) should have a file size of exactly 512 bytes.

Example 2: Back-up the MBR to a disk sector.

Start off just as with Example 1 and ensure that Readonly is unchecked. "Select All" from the Edit Menu and then from the same Menu choose "Copy". You now have the MBR on the clipboard so the next thing is to find a good sector on which to store the copy as a backup location. You might like to change the offset display to decimal rather than hexadecimal - that is up to you. You would do this from the "View Menu >> Editor Offset Format".

There are two sectors that are generally OK to use. Any sector might do of course but you dont later on want to overwrite anything of significance nor have your backup overwritten by something else. The last sector of Track 0 (sector 62) and the very last sector of the drive (the total sectors minus 1) are generally the best places. You can jump to the very last sector from the "File Menu >> Disk >> Load Last Sector" or you could move sector-by-sector to Sector 62 by holding down the <Shift> key and then  toggling the <F7> and <F8> keys. Check that all 512 bytes of the chosen sector are zeros or are all 0xF6; if not then desist. Check that you have only one sector loaded; if not then desist. Check that your position is where you want to be; if not then desist. You can then choose "Select All" and then choose "Paste" (though often just choosing "Paste" is enough). The zeros should now be replaced by the same code that was on the MBR but these changes are not yet commited. When you next try to move sectors or close TinyHexer or if you go to the "File Menu >> Disk >> Write Changes" you should be prompted to write the changes to disk along with a scary warning. At that point the changes will be commited if you say Yes. At that point you are on your own.

The advantage of using the last sector as opposed to sector 62 is that if the MBR is corrupt there is a reasonably good chance that the adjacent area will  also be bad and this could have rubbed out the backed-up data on the nearby sector 62. The disadvantage is really only that you should not use the last sector if you have NT-dynamic disks or any RAID arrays in place. This is because both use the end of the drive for their databases. The very end of the drive is nearly always outside any partitions, since it would be pure coincidence if the final partition's cylinder boundary coincided with the end of the hard drive. It can be handy to have the backup on the hard drive like this but it is always one step safer to have it stored as a 512 byte file on some removable medium.

Example 3: Restoring an MBR.

If the backup is on a file you must first open that file. An MBR backup should be 512 (0x200) bytes in size so the value of 0x200 should show in the bottom left of the status bar. Then from the "Edit" menu choose "Select All" and then from the same menu choose "Copy". The data should now be on the clipboard so you can close the window and open the correct Disk, whose MBR you want to restore, just as described at the start of Example 1. If the defaults were left alone just the first sector, the MBR, will have been loaded. From the "Edit" menu choose "Select All" and then "Paste" the clipboard contents. Now is the moment of truth. Double-check that you have chosen the correct Disk and that you have pasted the backed-up data to the correct Sector. You can scroll around, etc, etc, to see what is where. When you are completely happy go to the "File" menu and on to the "Disk" sub-menu and choose Write Changes.  Agree to do this and the backup  should be restored.

If you are backing up from a known back-up on a sector then it is basically the same. You would just need to navigate to the backup sector and copy it to clipboard. Go to the first sector of the relevant Disk and choose to paste the clipboard contents. Write the changes and agree to the warning prompt.

Example 4: Changing the Disk Signature or the Active Partition.

In the following diagram, the last 72 bytes of an MBR, (which incidentally the NT command fixmbr does not over-write so long as the last two bytes of the sector are 0x55 and 0xAA) the Disk Signature is displayed in red text and the four partition table entries have four different coloured backgrounds. If the first byte of the sixteen bytes making up each partition table has the value 0x80 then that is the active partition. Only one primary partition (excluding any extended partition) should be marked as active. In this example the first byte of the second partition table (blue background) is thus marked as the active partition.

Offset       0  1  2  3  4  5  6  7   8  9  A  B  C  D  E  F

0000001B0                            19 BB A2 BF 00 00 00 01
0000001C0   01 00 04 FE 3F 03 3F 00  00 00 C5 FA 00 00 80 00
0000001D0   01 04 07 FE FF FF 04 FB  00 00 3B 8B 38 01 00 00
0000001E0   C1 FF 0F FE FF FF 3F 86  39 01 35 B6 12 02 00 00
0000001F0   C1 FF 07 FE FF FF 74 3C  4C 03 20 DC 02 01 55 AA

The "soft" disk signature; 19 BB A2 BF (written in hex in red at offsets 0x01B8 through 0x01BB) can be changed easily enough. Just open the relevant Disk and scroll down to that position. Put the cursor at 0x01BB and confirm from the status bar that you have located the correct spot. It's hardly worth saving these four bytes to a file - we just write the four pairs of hex down as an eight-character string if we want to keep the value backed-up. To edit to a new signature of your choice simply enter the values you want and then from the "File -> Disk" Menu "Write the changes" just as in the previous examples.

The partition tables' active partition (the table's byte at offset E in each line) is also pretty straightforward to edit as is the type of partition format (the byte at offset 2 in each line). The types shown here are 04 (FAT16 >32MB), 07 (NTFS) ,0F (ExtendedX) and 07 (NTFS) respectively. We actually recommend that you use other utilities (such as PTedit by PowerQuest now Symantec) because they make the translation of the bytes into more meaningful values for us humans to better comprehend and more easy to edit correctly. Changing the format type, by the way, does nothing to actually format the partition. Most types can however be "hidden" by giving the first nybble the value 1 instead of 0. Thus 1F becomes a hidden NTFS partition.

The very useful PowerQuest/Symantec tools PTedit and PartitionInfo can be freely downloaded from the public FTP folder. One particular value of PTedit is that one can just as easily see and edit any logical partitions as one can the primary partitions. You will need to download the correct version for your operating system (DOS or Windows or NT-based Windows). We have also put together a pqutils.cab BartPE Plugin so that you can easily add these two programs to your BartPE CD.

Example 5: Quickly "Zero" a Hard Drive.

A brand new unpartitioned hard drive is normally completely full of zero bytes. However, if you want to "effectively" re-zero a drive, because you are in a hurry, then all you need to do is to replace the MBR code with zeros. What could be easier. Open the relevant Drive and put the cursor at the very first byte and hold down the 0 key until all 512 bytes are full. Write the changes and you have an effectively zeroed hard drive. Most software will not look beyond the MBR and so will behave just as if it is a brand new drive. Less complete, but quicker, is to just zero the 64 bytes that comprise the four partition tables. This is equivalent to deleting all paritions on the drive. If, on the other hand, you want to be even more sure then zero the first 64 sectors (the whole of Track 0's 63 sectors plus the first sector of Track 1). Zeroing all of Track 0 (which utilities such as BootIt-NG call the EMBR or extended master boot record) will fool just about all utilities and boot sector viruses. A proper full zeroing of a drive can have other benefits (such as re-writing the bad sector map or of ensuring that data retrieval is most most unlikely to ever succeed) but is really best done using the hard drive maker's own diagnostic utility. Zeroing the first sector or track may be reversible but complicated but zeroing a whole drive will destroy all the data on it.


Footnotes:

1. There are a couple of areas that one should be particularly wary about when using a disk editor as well as when booting to any NT-based or Linux operating system (from a hard drive or from a live CD).

  • Any hard drives containing DDO can behave quite abnormally when accessed from other than the hard drive. Such DDO overlaid drives behave in a proprietary fashion at the best of times. DDO was once commonly used on motherboards with older BIOSes and to which hard drives beyond the capacity of those BIOSes were attached in order to overcome those limitations. If you have a modern system that natively supports drives larger than 128GiB (137GB) then this shouldn't be an issue, since there would never have been any reason to install such overlay.
  • After the BIOS has chosen a boot device, both Linux and the NT-based OSes can poll the hardware for themselves. If the incorrect drivers have not been installed along with the OS then the OS may not see any such RAID/SATA/SCSI drives or, potentially more destructive, may see all the drives which had previously been incorporated into the sole array entity. In this situation anything written to any part of a drive can break an associated array. If you are using RAID then do take particular care and do install the correct drivers from a floppy diskette using the F6 option at the very start of the PE start-up either if such hard drives are missing or if they appear to be duplicated. If you are not using RAID nor NT Dynamic Disks then these adverse affects should not be an issue - though you may well need the drivers at startup in order to be able to access all the hard drives in the system.

Don't go down these avenues without first having any vital data backed-up offline beforehand. That, of course, should be a given all the time. However this is too seldom done - to the great profit and delight of professional recovery firms..

2. To run a program with admin status under Windows Vista simply right click on the application's shortcut icon (instead of double-clicking on it as normal to open it) and then choose "Run with Administrator Status".

3. Alternative Windows Disk Hex Editors:


Appendix: (Pre-compiled BartPE Plug-ins)

[Top of Page]  [Disclaimer]

Web design by paulski.com - last updated 28th February 2010
Pages best viewed using a CSS2-compliant browser such as Firefox or Opera
Valid HTML 4.01! Valid CSS!