[MM] Moving files

Joined: May 22nd, 2014, 6:11 am

November 20th, 2015, 8:40 pm #1

Alright, a lot of people have asked for this, so here it finally is. Beware though, I haven't tested this method fully out. It could potentially host some issues.

Also, this tutorial is not finished. It should however provide you with all of the information necessary to successfully move a file, even if all steps aren't explained in detail yet.
Software used in this tutorial:
  • HxD - hex editor to modify binary ROM image (the game data)
  • rn64crc - tool to recalculate the CRC checksums of a modified ROM
  • Decompressed Majora's Mask N64 ROM of choice (you can use ndec to decompress your N64 Zelda ROMs)

[+] Example of the process (simpler explanations)
In this example, we'll be moving Link's file to unused space at offset 0x03D00000. I've imported this sword into the file, so the size of it has increased:
[+] Spoiler

Thanks to CDi-Fails for providing me with this sword model!
If you want to follow the example, go ahead and download the .RAR file that is attached this post. It includes a clean version of MM Link's object file, as well as a modified one with the imported sword model. A decompressed, American NTSC 1.0 version of MM will be used in this tutorial.

If you compare the two files, you'll see that the modified one is slightly bigger in filesize than the other one. This means that we can't put it back at the original location of Link's object file, since it wouldn't fit.



Go ahead and open up both of these files in HxD, as well as a decompressed NTSC 1.0 MM ROM.



In the modified file, copy all data. This can be done by hitting CTRL+A on the keyboard to select the entire file binary, and then CTRL+C to copy that data.



Now switch over to the ROM. We need to go to the new location where we'll place the modified file. As stated above, we'll place it at offset 0x3D00000 in this tutorial, since data around that location isn't normally used by the game.

To go to a certain offset in HxD, you can etiher click "Search" and then "Goto", or hit CTRL+G on the keyboard. When you do either, a box will pop up where you can enter the specified offset that you want to go to. Enter "3D00000" and then hit "OK".



You will arrive at that offset almost immediately. Before pasting the file though, you should take a look at the offset.



You'll need to remember this offset, since you'll be needing new pointers to tell the game to load Link's file from this new location instead of the original one. Therefore, I recommend you to write it down in notepad or something similar, so that you won't forget it.



Now go ahead and paste write the data of the modified file at that location. It is very important that you "paste write" the data - don't paste it like regular. That would increase the filesize of the ROM, and it would become unplayable.

You can paste write through simply right-clicking and clicking paste write, or hitting CTRL+B on the keyboard. Remember to always and only "paste write" when altering the data of Zelda ROMs using HxD, unless you're very sure of what you're doing.

When this is done, look at the offset you arriven at.



You'll have to remember this offset as well. The previous offset that you documented will tell the game from what location to start reading the file, whereas this one tells the game where to stop reading it. Write it down as well, so that you'll be able to create the new pointer.



Creating a new pointer is actually really simple. Just take your start offset and attach your end offset to it.



Now you've successfully moved the file! The game doesn't magically know this though; you'll need to rewrite the pointers that tells it from where to load Link's object file.

To do this, first you need to go back to offset 0 of the ROM.



Now switch over to the clean, unmodified version of Link's file and copy all of it just like you did with the modified one.



Switch back to the ROM. We're going to search for the entire file, to find the location in ROM where it's originally stored.

To search for specific data in HxD, you can either click "Search" and then "Find", or simply hit CTRL+F on the keyboard. Just like when going to an offset, a box where pop up. Here you can enter the data you want to search for; just paste the data you copied from the unmodified file as this. Since the data you copied was so large, it won't actually display what you entered. This doesn't matter though, as HxD still will search for the the stream of bytes that you entered.

Before you search though, you'll need to change the "Datatype" from "Text-string" to "Hex-values".



Click "OK", and you'll arrive at an offset after some selected bytes. These bytes are the data you searched for, or in other words: Link's object file. Now you can see from what offset the game originally loads Link's object file from, by looking at the offset where the marked bytes start.



Write this one down in notepad as well, as we'll be searching for this offset in ROM to locate the original pointers to the file.

Notice though, that the offset displayed in HxD (like in the picture above) only consists of 7 digits. Data is represented by bytes in HxD, which in hexadecimal consist of 2 digits. Since 7 is an odd number while 2 is even, the offset can't be stored in bytes exactly as displayed above.

To solve this minor issue, simply put a 0 at the start of the displayed offset, to make it consist of 8 digits (an even number) while still keeping its value.



Now it's time to locate the pointers in ROM (there are three of them that we need to replace, to be exact). This isn't very hard, just search for the start offset of the file that we located previously.



You should arrive at an offset displaying the exact same bytes as you searched for.



Here, paste write the entire new pointer that we created earlier, which points to the new location in ROM where we put the modified file.



Now we've replaced one of the three pointers. To find the next one, hit the F3 key to "find again", which will search for the same data again. Now you should arrive at an offset displaying another pointer to the original file.



As you can see, this pointer is actually right next to the first one we replaced.

You can't replace this pointer with the entire new one though. This specific pointer actually only points to the start of a file, so just paste write the start offset of our modified file here.



Now there's only one pointer left to replace

Hit F3 again, to search for the pointer (to the original location of Link's object file) yet another time. Once again, you should arrive at an offset displaying the exact bytes that you searched for.



This pointer is like the first one, so you can go ahead and replace it with the entire new pointer that we created before.



Now we're done with HxD, and the "hard" parts of the tutorial are over. Nice job! Go ahead and save the ROM before closing the program.

We're not done yet though. If you were to load the game in an emulator right now, you'd get an error and be unable to play the game. This is because the CRCs (basically a type of security checks built in to the ROM) of the game have been corrupted, which happens when modifying certain areas in ROMs of N64 games.

To solve this, we'll need to use the program "rn64crc". To run it, open up command prompt and simply drag and drop the program onto the command line,



Now hit space, write "-u" and then hit space again. This tells rn64crc to update the CRCs of the ROM that you'll provide it with soon.



Now drag and drop your modified ROM onto the command line.



Now simply hit enter, and the program should finish the process of recalculating the CRCs in our modified ROM!



Now we're done! Open up your ROM in an emulator, get the Guilded Sword equipped and you'll see the result of your moved file!
[+] Spoiler
Once again, special thanks to CDi-Fails for letting me use his sword model!
[+] WIP General explanation of the process (overly detailed, but hopefully well-explained)
Introduction:
First of all, why would you even want to move a file in the first place, and when would it be useful?

Well, sometimes while modifying individual files extracted from the game (like the file containing the mesh and texture data for Link's model( his "object file") for example), you might want to add something to it (like importing a new sword model into Link's object file).

This will increase the size of the file though, so you won't be able to "put back" the file at its original location in ROM, since it simply doesn't "fit" there with the increased size of the file. This is a typical situation in which you'll have to move the file.

Finding the pointers:
If you want to move a file, it is vital to know at which offsets in ROM its entries (pointers) in the file tables are stored. This is because these tell the game from what offsets (start to end) to load a specific file, which essentially is what we need to modify when moving a file.

So, in MM there are 3 pointers that you will have to replace when moving a file. Finding these shouldn't really be a big deal, you'll just need a (unmodified) copy of the file you want to move, an MM ROM (version of choice) and a hex editor (preferably HxD).

Fire up your hex editor and open up your ROM and file in it (a clean version of the modified file which you're intending to move).

In the unmodified version of the file you're moving, hit CTRL+A and then CTRL+C to copy the entire file binary.
Switch over to the ROM. Now hit CTRL+F to search in the ROM.
In the "Search for" box, hit CTRL+V to paste the binary of the file you copied.
Set the "Datatype" box to "Hex-values" and hit "OK" (this will make it so that HxD searches for hexadecimal/binary numbers instead of other types of data).

You should arrive at a result almost immediately. From this you can easily see the start- and end-offset of the file, however you'll actually most often just need to know the start-offset of a file to locate its pointers in the file table (a chunk of data that tells the game where all individual files are stored in ROM, through displaying their start- and end-offsets)

Before doing anything else, I should explain the "formatting" of the pointers we're replacing:
[+] Spoiler
1 - xx xx xx xx yy yy yy yy
2 - xx xx xx xx 00 00 00 00
3 - xx xx xx xx yy yy yy yy

xx = Start-offset of file
yy = End-offset of file
Now, to find these, just hit CTRL+F and search for the start offset of the file you found.
A result should pop up - this is the "1" pointer.
Next to this should be the "2" pointer".
To locate the "3" pointer, simply hit F3 to "Find again".

Write down the offsets of these somewhere, so that you won't forget them.

Moving the file and re-pointing:
After you've found the pointers, the only things left to do is to actually move the file and then re-write the pointers according to this new location.

To do this, open up the file you want to move in HxD and hit CTRL+A and then CTRL+C to copy the entire binary.

Switch over to the ROM and decide where you want to move the file to. You'll probably want to move it to space that's not already being used by the game. Everything after around offset 0x3D00000 is free space in most Zelda 64 ROMs - we'll use that in this example.

So hit CTRL+G and write "3D00000" in the "Offset" box to go to this offset. Here, simply paste your file in (CTRL+B).

Now it's time to "create" a new pointer so that the game will load the file from this new location instead of the old one. To do this, simply take the start- and end-offset of your moved file and put them together.

For example, if the start-offset of your moved file is 0x03D00000 and the end-offset is 0x03D1E250, the new pointer will be 0x03D0000003D1E250 (the prefix "0x" just signifies that a value is hexadecimal).
So go to the offsets of the "1" and "3" pointers you located earlier and paste this there. But what about the "2" pointer?
As you might remember, this only points to the start of the file, so you should only write the start-offset of your moved file there (would be 0x03D00000 in this example).

Now you're done with this part of the tutorial, so you can save the ROM and close HxD. If you were to open your ROM in an emulator now though, you'd get an error, even if you did everything completely right.

This is because the N64 has a sort of built in security check to prevent modification of its games, called CRC checksums. I will not delve too deeply into what these actually are (as I'm not completely sure myself), but know that by modifying certain areas of ROM they will become corrupted and the game won't run (which is exactly what happened in our case).

We will have to update the game's CRCs in order to fix this issue.

Recalculating the CRC:
The last step is to recalculate the CRC - explanation will come sometime.
MovingFilesTutorial.rar (91.28 KiB)
Last edited by Ideka on June 10th, 2016, 9:31 pm, edited 36 times in total.
"What?! You don't have my money?! How dare you! You'd better bring me my money...or else!"
~ Happy Mask Salesman

Youtube
Webpage
Quote
Like
Share

Joined: October 26th, 2015, 3:54 pm

November 20th, 2015, 8:55 pm #2

I don't know if this tip applys to majora's mask but in oot. If you search for the start and end addresses of the file your replacing you can repoint those. It sounds faster but I could be wrong.
Last edited by SoundBlitz on November 20th, 2015, 9:09 pm, edited 1 time in total.
Mr R&D
Quote
Like
Share

Joined: May 21st, 2014, 6:43 pm

November 21st, 2015, 4:03 am #3

I feel like this tutorial could be formatted into steps and written a lot better. A big chunk of text is meaningless to newbies such as I. Along with that, I have some questions.

1. "Fire up your hex editor and open up your ROM and file in it. In the file you want to move, hit CTRL+A and then CTRL+C to copy the entire file binary. Switch over to the ROM." I don't understand this at all. Aren't we already IN the ROM? It says "Switch over to the ROM" but in the beginning it says open up your ROM already.

2. What is a DMA?

I think that's it, considering it's hard to follow from the start. :/
I don't stand stupidity.
Quote
Like
Share

Joined: May 22nd, 2014, 6:11 am

November 21st, 2015, 7:36 am #4

Sorry about that. This tutorial is not finished, but (eventhough it might be confusing) it should give you all the information necessary to be able to move a file. Therefore I figured that I could as well post a WIP version of the tutorial instead of releasing the full version much later. But yes, improvements are to come soon, I intend for this tutorial to be as noob-friendly as possible!

@SoundBlitz:
That's essentially what we're doing here, but instead of searching for both the start- and end-offset, we're just searching for the start-offset of the file. This is because one of the pointers that need to be replaced just points to the start of the file and doesn't include the end-offset.
Last edited by Ideka on November 21st, 2015, 7:57 am, edited 1 time in total.
"What?! You don't have my money?! How dare you! You'd better bring me my money...or else!"
~ Happy Mask Salesman

Youtube
Webpage
Quote
Like
Share

Joined: October 24th, 2015, 6:13 pm

November 21st, 2015, 8:08 pm #5

I agree with Jabu in relation to creating it to be more userfriendly, while also being informative. Here is something I sent to a friend recently pertaining to the F3DZEX page on CloudModding just to give you a frame of mind of what is wrong with tutorials and guides in zelda communities (and others even outside of Nintendo 64). I've been also updating pages there in a hopefully attempt that it's of much avail to someone.
vexiant wrote: If you think you're up to the challenge, I'd say you should make a tutorial displaying how you can use the information displayed on the F3DZEX page in a practical manner. All the while, have it be informative, elaborate as to how and why things work a particular way, etc etc. Most individuals who concoct tutorials neglect relating to individuals by either using superficial wording, or pointless jargon that can just as easily be put in parentheses (so that they're not completely ignorant to the fact of the process). I know this first had, as I was like this upon first entering the scene: Most noobs do not learn anything you tell them (as it's usually not put properly anyways, as stated above), but usually instead merely copy the process behind how to do said "tutorial." That of course renders their 'newly acquired knowledge' useless. Moreover, when I joined, I wasn't fortunate enough to be around the plethora of tutorials and guides that are now present, but I did befriend Flotonic, Twili and others early on, and still failed to wrap my head around everything. Granted, I was hardheaded then, but the primary issue, in my opinion, was lack of "good" quality in tutorials which are still present today.

With that said though, I'm no teacher myself. Teaching is much like that of leadership; it's a practical skill that not every "Joe" can master and can be hard to come by. I study philosophy and have read Plato's Republic, but I kind of put two and two together out of my own personal introspection. Nonetheless, do or don't, you're doing an excellent job. Keep up the good work!
Last edited by vexiant on November 21st, 2015, 8:10 pm, edited 1 time in total.
you need advanced hexes and asm to hack zelda
Quote
Like
Share

Joined: July 6th, 2014, 8:50 pm

April 28th, 2016, 7:41 pm #6

This is a great WIP tutorial.
I can't wait until it is finished. :)
this will help me with my mm 3d link port.
Keep up the good work. :)
OMG it worked this method actually worked!
It took me a hard time to understand this tutorial but I finally got it down you did it man you found a working free space method!
If you don't mind @ideka.
I would like to rewrite out this tutorial finishing it if you don't mind. And thanks for finding this method I've been looking for this for a long time. :)
Last edited by zelda5655 on April 30th, 2016, 1:30 pm, edited 1 time in total.
surely you'll be able to get the Majora's Mask back believe in your strengths believe Happy Mask Salesman of Zelda Majora's Mask.
Quote
Like
Share

Joined: May 20th, 2014, 7:11 am

April 30th, 2016, 1:48 pm #7

Dunno why the tutorial is tagged [MM], OoT uses the same file system
Quote
Like
Share

Joined: May 22nd, 2014, 6:11 am

April 30th, 2016, 2:22 pm #8

@zelda5655:
I'm already in process of writing a simpler version of this tutorial, as well as cleaning up and finishing the existing one. Please don't write your own tutorial of this very same process; it'll just make things less organized and will probably also confuse people.

Actually, I guess you can go ahead and write your own tutorial. But I'd prefer if you post it in this topic, just to keep the forums more organized. Good luck!

@mzxrules:
The process of moving files in MM seems to require more steps than that of OoT, in my experience.

In OoT, I've only ever needed to replace one single pointer (namely in the DMA table, I believe) when moving files. In MM, however, the game crashes on me unless I replace all of the three pointers discussed in the above tutorial.

This same process could potentially work with OoT as well, I guess. But since there already is a simpler method for that game, I chose to tag this tutorial specifically with "[MM]", to avoid confusion.
Last edited by Ideka on April 30th, 2016, 2:59 pm, edited 4 times in total.
"What?! You don't have my money?! How dare you! You'd better bring me my money...or else!"
~ Happy Mask Salesman

Youtube
Webpage
Quote
Like
Share

Joined: July 6th, 2014, 8:50 pm

April 30th, 2016, 2:53 pm #9

ok I will leave it to you then. :)
surely you'll be able to get the Majora's Mask back believe in your strengths believe Happy Mask Salesman of Zelda Majora's Mask.
Quote
Like
Share

Joined: May 20th, 2014, 7:11 am

April 30th, 2016, 3:29 pm #10

I don't see how your "simpler" method is very practical Ideka. If you're only replacing "one pointer", then that means you're just moving the file to a different rom address without resizing it.
Last edited by mzxrules on April 30th, 2016, 3:29 pm, edited 1 time in total.
Quote
Like
Share