Kevin Vance - Last night, while working on dsmzx, I got the Caverns title screen…

Entries | Archive | Friends | Friends' Friends | User Info

10:45 pm

Sunday, August 26th, 2007
Previous Entry Share Next Entry
Tags, ,

Caverns title screenLast night, while working on dsmzx, I got the Caverns title screen to show up by accident. I tweaked the startup code to switch to the config directory at startup to read a configuration file, and since CAVERNS.MZX was in there too, it started!

Yesterday was a tough day for dsmzx hacking. I realized that my copy of mzx2.81g was really old, and it took hours to port my changes to version 2.82... especially changing from one custom make system to another similar BUT DIFFERENT custom make system. I never thought I'd *want* to see autotools so bad. Now I'm up to date, and I'm using git-svn to sync things up. git is a fantastic source code management tool. Being able to merge branches fast and effortlessly... wow. Goodbye SVN for future projects.

The next problem was that MZX would always crash a few seconds into the file selection dialog. Turns out that I only have 16K of stack space to work with, and MZX does a fair amount of stack allocation. Allocating the world structure in main() killed the first 10K of my stack before the program starts! I started using libnds's default exception handler, providing me with a red screen of death when things like this happen in the future.

But hey, I got it running and at that point I had been doing this all day. I tried to continue my BioShock game, but my 7900 was running too hot. ASUS really needs to get back to me about that...

Caverns game menuThe next day came dsmzx user input. Fortunately, my up-to-date copy of MZX actually has the input code for PSP and GP2X, so it should have been easy. But it turns out that ndsSDL is half-baked. Or it's a quarter baked. I had to remove some extra libnds calls (ndsSDL assumes that it gets to monopolize libnds) and modify the magnitude of the fake "joystick axes" it returns. Until this point, I was not initializing the SDL video subsystem, since I had to roll my own. It turns out that the event subsystem requires the video to be initialized, or the event queue can never accept input. Lame.

But now I can play the first few boards of caverns! It runs at an acceptable speed, with palette changes causing slowness. I think this is MZX doing something inefficient, since running the palette updater on every frame had no effect on my test app. There is also a noticeable amount of tearing. I can't waste my last two 128K banks of VRAM on double buffering, since the only other sprite VRAM bank for the subscreen is only 16K—not enough for a full charset on the subscreen. Maybe it would be acceptable to work on an offscreen buffer and DMA it into VRAM on the vertical blank.

The next game I tried to run (xenogenesis) triggered a stack overflow on the second board. I have more work to do. And using the editor kills a lot of stack. A great way to check for functions that do too much stack allocation is objdump -S and search for sp, #[0-9]\{4\}. It returns lines like:
 2073470:       e24ddc47        sub     sp, sp, #18176  ; 0x4700
That's almost 18K of stack allocation in one function!

I tried getting sound to work tonight, but as you may have guessed, ndsSDL's audio subsystem is a trainwreck. It triggers the hardware exception handler, but prevents it from writing precious debugging information to the screen. SDL is (annoyingly) deeply integrated into MegaZeux, so I'm not sure whether I should try to work around SDL at this point, or try to fix it.
Link )Reply )

[User Picture]From: kevincarter
2007-08-27 04:41 am (UTC)
That's really cool-- your technological knowledge of this is incredible. I'm starting to think about moving from doing sysadmining into coding. Any suggestions on where to focus?
(Reply) (Thread)
[User Picture]From: kvance
2007-08-27 06:41 pm (UTC)
It's just so vast. Here's what I can think of for a good foundation, with suggestions for each topic in ~descending order of importance:

Learn the common languages: C, C++, and Java.
C has a lot of downfalls, but it's fast and pretty much everyone knows it. Most Free software is written in C. You should be able to create and dereference pointers, lists of pointers, pointers to pointers to arrays of pointers, etc. Try finding potential buffer overflows in your own programs, or others'. Use valgrind to find memory leaks.
C++ is the opposite of C in terms of simplicity, but pretty much every software company who needs performance uses it. Classes, inheritance, scalar vs. array allocation, constructors/destructors, templates, STL, Boost.
Java is bloatier than C++ in some ways, and more streamlined in others. If your company isn't using C++, it's probably using Java. Much of what you learned about C++ applies.

Learn about how things work at the machine level.
Assembly code is still good to know for when you're debugging. Try porting a C program to asm piece by piece. Try writing programs for old computers like the C64, which have a simpler instruction set. Read The Art of Assembly Language.
Understanding how the CPU works is good for the same reason. Write a cycle-accurate emulator for a simple (or fictional) CPU. Read Computer Architecture: A Quantitative Approach.
Try breaking the copy protection on an old game, or reverse engineering a data format using an assembly level debugger. The same techniques will come in useful when your own code is failing in confusing ways.

Learn about algorithms and efficiency.
Write a recursive algorithm, try flood-fill first.
Learn Big O() notation. Write a linked list, a binary search tree, and a hash table, and know when to use each. Have CLR on your bookshelf as a reference.
(If you're interested in games, write a binary space partitioning tree and a quadtree later.)
(If you're interested in filesystems and databases, write a B+ tree later.)

Use higher level languages. In most cases, the execution speed to programming speed tradeoff is worth it.
Perl, ruby, and python are all popular. Python is my workhorse for 90% of my new projects.
Write a non-trivial program in LISP. I haven't touched on functional languages so far because they're so unpopular. But it will change the way you think about programming.

I'm sure there are gaping holes in this, but it's all stuff I use in my daily life. Wikipedia is usually an adequate starting point for any programming topic.
(Reply) (Parent) (Thread)
[User Picture]From: kevincarter
2007-08-27 07:45 pm (UTC)
This is really awesome advice-- thanks! I know what I asked was kind of a ridiculous question, but you summed up exactly what I was looking for. Thanks again, seriously.
(Reply) (Parent) (Thread)
From: zixyer
2007-08-27 05:51 am (UTC)
I ran into the stack issue with ffmpeg. I remember it being really frustrating to debug! The shocking part was that I didn't run into it sooner, what with all the long chains of function calls ffmpeg makes.
(Reply) (Thread)
[User Picture]From: kvance
2007-08-27 04:31 pm (UTC)
There's a trick where you can use a linker script to remap the stack right above the main RAM, so the stack overflows onto the heap, hopefully causing no problems if you're not using too much memory. I haven't tried it yet, but you probably can't use memory protection when doing this.
(Reply) (Parent) (Thread)
From: zixyer
2007-08-28 10:05 am (UTC)
Hmm, I think devkitpro is set up so both stacks is in some fast memory that is dedicated to one processor. Anyway, without threads or recursion (which nobody really ever uses in C) it's pretty easy just to go through and make the particularly big automatic variables global.
(Reply) (Parent) (Thread)
From: c99koder
2007-08-27 09:26 pm (UTC)
Your DS LED cover is broken :P

Looks pretty cool, glad it's working out. The sound code in DreamZZT is based on the libsdl arm7/arm9 audio code -- it's a bit flakey, and I think it's the source of the random freezes. The arm7/arm9 wifi code is just as flakey, I wish they'd improve the communication between the two CPUs.

How do you use the built in exception handler? I've found it'll only show me the exceptions if I have a libnds console on one of the screens -- is there another way to do this? My console kicks their console's ass, but it doesn't get the exception output :P
(Reply) (Thread)
[User Picture]From: kvance
2007-08-27 09:42 pm (UTC)
The whole hinge is broken. Notice how I'm propping up the top screen with my hand or the mouse? I have a new Lite for actually playing games on.

I didn't have to do anything to get the default exception handler to work. It should set up the default console on its own. Though recently, it hasn't been able to print anything to that console on my dsmzx builds. I'm not sure why...
(Reply) (Parent) (Thread)
[User Picture]From: price
2007-08-27 09:42 pm (UTC)

Fuck AD! *cheshire grin*

It's amazing how a bunch of teenaged douchebags can turn into normal human beings. I still chuckle at how the high school drama played out over IRC. At least I can laugh at it now.
(Reply) (Thread)
[User Picture]From: kvance
2007-08-31 02:34 am (UTC)
Heh. I must have gotten out at the right time. I always hear about this drama, but I can never remember any of it :P
(Reply) (Parent) (Thread)
[User Picture]From: dariusk
2007-08-29 02:54 pm (UTC)
That is amazing! You even inspired to me to post a massive nostalgia trip on my games blog.
(Reply) (Thread)
[User Picture]From: kvance
2007-08-31 02:33 am (UTC)
Awesome! Welcome to my friendspage, btw :)
(Reply) (Parent) (Thread)
From: ex_md744
2007-08-29 07:25 pm (UTC)
Awesome. You sure you don't want to hack Japanese visual novels for me? :P

09:43 @md> hah
09:44 1> Heh, msx.
09:44 2> i don't get it
09:45 3> Emulating an msx?
09:45 1> Yeah, that what it looks like.
09:46 @md> 2: Friend of mine ported mzx to DS
09:46 2> why can't the DS emulate somthing useful.. like the PS3
09:46 2> ohhh
09:46 2> make him port onscripter

I was surprised one of the guys in our channels almost knew what it was called.

If you ever get ndsSDL fixed nicely, try and get onscripter to compile, huh? :P
(Reply) (Thread)
[User Picture]From: kvance
2007-08-31 02:53 am (UTC)
I doubt I'll ever have enough free time to. A port would probably be possible, but annoying if it doesn't support 256x192, and really annoying if it can't do less than 512x384.
(Reply) (Parent) (Thread)
From: ex_md744
2007-08-31 03:03 am (UTC)
ons is pretty nice about laying out text if you set the window right.

There was actually a port already to the GBA, and it was perfectly playable.

and really it's just a joke, I don't think we care that much about a DS version. :P

I'd rather have you reverse engineer the script's op-codes (totally different game) so I could change the length of a string. Trying to translate in exactly the same distance sucks.

I should stop trying to pull everyone I know into this group I guess?
(Reply) (Parent) (Thread)