Kevin Vance (weblog of)

Thanks everyone for poking at my new site code! Especially thanks to dariusk for finding the broken icons in the RSS feed, and honorable mention to rspeed for attempted javascript injection in the user comments.

I'm not sure how long it'll take me to write all the text I have to write. Hopefully by the weekend. This has been interesting, especially the non-CSS parts, but I'm looking forward to spending my side project time on something that's more fun. I haven't forgotten about you, Corner Office!

Finally finished writing the code for my website. If you have some time, please click around devg.kvance.com and let me know if anything is wrong in your browser. I've used at various points Firefox, Opera, Chromium, Safari, and IE8 to check it but not very thoroughly.

Things it does:

• Boring plain text
• Detailed index
• One-liner index
• Project view w/ summary, comments, screenshots, download pages
• Multiblog w/ RSS feed

Especially try leaving comments, downloading files, and using the multiblog RSS feed. The content is all placeholders, and the devg site will be going away whenever I get the real content added.

I've been playing around with firefox 3.5 recently (I got RC3 a few days ago).

The changes with the most impact for me are in the session store. I like that it lets you choose which windows to reopen after restoring from a crash, but there is still that "jeopardy moment" where you can press a button to destroy the old session, kind of like when pine asks you if you'd like to delete your old mail at the start of the month. The last version of firefox used a regular dialog box for this, so I'm sure I'll get used to it.

It also restores windows from other virtual desktops by minimizing them instead of dumping them all out to your current desktop. That's an improvement, but still not right!

The speed increase is quite noticeable to me (running Linux on an Intel Q6600) both in loading the initial session store, and watching text instantly reflow while resizing windows. The GUI is still not very responsive while it's loading ~70 tabs, but at least it doesn't take so long now.

I also crashed it once by clicking on a link from kartos over AIM. I'm going to blame the flash plugin for that because I have no reason not to :P

duinlas just told me id software was bought out. The only big independents left that we could think of were Valve and Epic. And Epic doesn't even make PC games anymore! I guess I've been watching this era end for a long time but still :(

I had this weird recursive dream today. Since I only slept ~4 hours, I decided to take a nap around noon. In the dream I would "wake up", notice something wrong/inconsistent, figure out that I must still be dreaming, "wake up" from that, and so on. At one point during the loop, I realized that since I was in a dream, I should be able to control it. And I did! I made everything disappear, and started running really fast. It only lasted about 10 seconds, but that's the only "lucid dreaming" I've ever experienced.

The loop reasserted itself after that, and eventually I woke up after what felt like an entire night of recursive dreams. The clock said I had been asleep for about 5 minutes.

Since I picked on Objective C earlier, it's only fair that I now call out python on a serious standard library shortcoming.

Today I noticed that the timestamps were wrong on the twitter posts I pulled from my twitter RSS feed. They appeared to be in UTC instead of local time, so I figured it would be an easy one-liner to convert them.

Not so. While python does have support for time zones throughout its date/time functions, it's up to you to actually build the timezone data yourself, parsing an environment variable or the /etc/localtime file!

Luckily, there is a third party library called dateutil that will create timezone objects for you. That functionality not being in the standard library is just baffling to me.

At least now, I have some semi-sane code to take a UTC time stamp from feedparser and convert it to a local time stamp for mysql.

    # Convert from UTC to local time.
    utc_tzinfo = {'tzinfo': dateutil.tz.tzutc()}
    local = dateutil.tz.gettz('America/New York')
    published = datetime.datetime(*feed_entry.updated_parsed[:7], **utc_tzinfo)
    localdate = published.astimezone(local)

    # Strip the time zone info to appease mysql.
    naivedate = datetime.datetime(*localdate.timetuple()[:7])

My personal website is long overdue for some work. While it is mostly dynamically generated, that generation comes from a handful of perl CGI scripts, rewrite rules, and (thankfully, at least) HTML templates. The problem is that if I want to add new content, I have to use the mysql command line to manually add it! This has dissuaded me from updating the site with little programs, etc. that should be in the archive.

I'm also extending the site to aggregate my various bloggy things into one page. I've been using twitter a lot more than LJ lately, so the LJ front-end currently on kvance.com isn't doing much good. Once that's set, it should be pretty easy to put out a complete RSS feed, maybe filtered over service, tags, or other parameters.

I'm doing the work with the django framework, which just for starters automatically gives you an admin interface based on your data model. And it's in my favorite programming language, and seems to be pretty popular, etc. Every once in a while, I'll poke around at some python web frameworks (I've used TurboGears in the past) but I seem to be making pretty good progress here.

I have it spitting out very plain views for projects, file lists, screenshot lists, and the multiblog. And I have a script to pull entries from twitter and LJ. Soon we come to my least favorite part: the actual web design and horrible, horrible testing process to make it work in every incompatible web browser. I heard somewhere that IE6 was dead, so nobody tell me otherwise okay? :P

26th birthday! Bacon for breakfast, hacking during the day, steak dinner tonight... then starting a new diet and exercise regime tomorrow :P

I can't remember, do I use this LJ for anything besides complaining about games not getting a PC port?

I'm glad Tim Schafer finally got Brütal Legend published, and I look forward to playing it some day. On a computer.

So in most sane programming languages, it's pretty easy to manipulate a list of integers. In python, for example:

Tags

:

nintendo ds

My NDS texture compressor is now available for download. It requires python 2.5 and a laundry list of python libraries. There's also a simple test program that lets you pan around the compressed image on NDS hardware/emulators.

It's generating okay output I think. On the standard computer graphics Lena image, compare the original and compressed versions.

It's incredibly slow though. Take a look at these timings for the 512x512 Lena image:

Configuration	Time (MM:SS)
Core 2 Quad Q6600, Python x86-64, Linux	3:59
Core i7 920, Python x86, Windows	2:39

There's a lot of places you could optimize this. Obviously, moving from python and numpy to C and assembly would help tons. The depth crushing and dithering could use fixed point instead of float, and so could k-means. And the final pass where we eliminate egregiously bad blocks is really bad. It runs PIL's color quantizer 1-3 times for each palette for each bad block! Not feasible for the entire image unless you have a nice supercomputer under your desk. That should probably do some kind of nearest-neighbor search for decent palettes to try.

I'd worry about image quality before optimization though. By default, we only use 4-color palettes since 2-color ones never seem to have good results. There are a lot of blocky artifacts. Is there an algorithm to reduce that? (Error diffusion over the image gradient on block boundaries?) Python is much more useful for that kind of exploration. The image dithering also may be causing more harm than good, since it's adding noise. Perhaps only dithering where it's really necessary, or applying some kind of adaptive blur would help in this area.

Tags

:

nintendo ds

As promised: a long, boring article about Nintendo DS texture compression. This article describes the compression format. I may follow it up with another one about how my compressor works specifically.

Overview

Hardware texture compression on the NDS allows you to display very large textures for the device. You can store two compressed 1024x512 textures on a device with a total screen size of 256x384. There is no performance penalty, as the hardware can decompress them on the fly. Unfortunately, the tools to generate compressed textures are not available to the homebrew community.

Compressed textures have a slightly complicated format, and the hardware is not very forgiving about how they are stored. The texture is divided into three parts:

• Pixmap: The pixels of the texture, grouped into 4x4 blocks
• Index: A map from each pixel block to a part of the palette
• Palette: The list of colors, used in groups of 2 or 4

The pixmap can be stored in texture slots 0 or 2. The index must be stored in slot 1. This prevents textures larger than 1024x512, since the index would be in the middle of them. Palettes are stored normally.

( Read more... )

Tags

:

nintendo ds

( Play me a victory fanfare... )

That's the output of my texture compressor running on an emulator, compared to the original. It also works on the hardware. The result is still rather blocky, but I have an idea of how to make it look better... hopefully not requiring a small cluster to do the calculations...

Anyway, this is the first non-trivial image compressor I've ever done. It's slow and the results are not that great yet, but I am feeling pretty good about it. According to my git logs, I started this last Tuesday evening. If I keep getting into these side projects, I'm never going to get any work done on that simple little game I was trying to write :P

Dear LJ,

IOU 1 long, boring article about NDS texture compression that nobody's going to read.

-- kvance

Tags

:

corneroffice, nintendo ds

So much for doing Corner Office in a month, making frequent LJ entries about it! I lost a bunch of days not doing anything on it, but I've been making progress again since last week (starting with the great log entry: "Wrote a bunch of code, not much of it works"). Since I started, a new devkitARM, libnds, etc. have been released. It was fairly easy to update to. I think the only difference was that some register names changed.

I also decided on how to package the game: a single .nds file, with the assets inside the internal .nds filesystem. The EFS library makes this possible, taking care of working on an emulator or actual hardware (by scanning your entire filesystem to find the .nds file!) Unfortunately, the EFS library, the EFS patcher, and libfat all had bugs. Workarounds have been posted on the gbadev forums, but I still lost a few days getting it all to work.

Also, I now have to have a loading splash screen. The first time the game loads, EFS has to scan the card for the game's .nds file since most (every?) loader doesn't fill in argv. Without a loading screen, that is a scary amount of time looking at a blank screen. After that, the game patches itself so it knows where to look next time. This is why EFS would be awesome if it worked out of the box.

For more exciting stuff, I finished the command GUI visuals. Every character in your party has a stat box on the subscreen, which can inflate and deflate to show that character's commands. The inflation, deflation, and the stat bars are all smoothly animated. I think the last prototype I need to do is to combine this command screen with the world view to have a little test battle. Then I can start cobbling it all together and making the levels.

The new computer I built in January can be a bit.. quirky, to put it nicely. One quirk is that after rebooting, the power LED will blink as long as the computer is on, until the power is cut to the motherboard. That was supposed to be fixed in the BIOS, so I attempted to upgrade.

That's when I found out another quirk is that if you press the wrong button during BIOS bootup, it can select a broken, preloaded overclocking profile that creates keyboard/mouse lag for DOS and other programs that interact on a BIOS level. Another side effect of this is to make flashing the BIOS impossible.

There is a long thread about this on the motherboard manufacturer's forum, which eventually has you pull the BIOS battery for a while to reset whatever's gone wrong. I tried this earlier in the week with no luck. It booted up to the same lag and the same inability to flash the BIOS. I even tried booting a WinXP PE CD to run the windows flasher, but it just pegged the CPU until I killed it.

Today I tried again, this time disconnecting everything from the motherboard except the video card, a USB keyboard, and the USB stick with the BIOS flasher on it. I also made sure to erase all of the preset overclocking profiles. Success! No more keyboard lag (now I can use grub again!), no more power light flashing.

While this hasn't affected everyday use, it has made me more appreciative of the system I built last year using established technology rather than the bleeding edge stuff.

Tags

:

corneroffice, nintendo ds

Here's a blurry video of my effort to combine the text printer and 3D world I was working on earlier. The interface needs a bit of prettying up, but I'm happy with the fade/slide in for the text box (and it's so easy to do using hardware fading and scrolling). I'm currently calling the game Corner Office, since I think just about all of it is taking place in an office building. So my LJ entries about it are now tagged corneroffice.

Tags

:

corneroffice, nintendo ds

I got a bit sidetracked on my DS project, but still making some good progress. The 3D world now renders correctly on the hardware. The sprite was showing up as all black with a transparent background, or color with a black background depending on whether or not I used the DECAL shading mode. Turns out it needed a fake material (emit=100%) and a normal vector.

You can also move around, collide with walls, and collide with sprites. These are using simple bounding box/bounding circle tests. A big box for the room bounds, and smaller boxes for obstacles. And circles for sprites. I also have an additional circle called the "interaction radius" where something can light up on the GUI and you can press a button to interact with it. This will be useful for interacting with sprites, and environmental features.

Then I started getting off track... first of all, I could not find a way to convert 8 bits/channel color images to 5 bits/channel (NDS format) using GIMP, ImageMagick, or PIL. So I wrote my own converter in python, using floating point and floyd-steinberg dithering. It's incredibly slow, but that doesn't really matter. The blue background of the example image has been run through that program.

Then I started looking at compression, since the output of my image depth crusher is in 16bpp NDS framebuffer format... very large. I originally used the LZ77 decompression routines that are conveniently in the BIOS. But unfortunately:

1. They only seem to output to VRAM or WRAM, which is inconvenient.
2. I switched emulators to no$gba in wine, which doesn't emulate that without a copy of the NDS ROM.

So I ended up with zlib instead. Despite some intimidating documentation and header files, there is a handy uncompress() function that does all the work. So that background image is zlib compressed framebuffer data, while the font is a zlib compressed PCX.

The font stuff is an extension of my earlier HexxagonDS text renderer. I nudged it over to C++, added some basic line-wrapping, and now it can draw 2-color fonts (an outline and a fill color).

Next: putting these together to have a 3D demo where you interact with stuff via text boxes!

Tags

:

corneroffice, nintendo ds

I've been playing around with a new DS project, using a little more 3D than I have before. That means even more time spent in Blender, a program one would think I'd be better at using my now. At any rate, I now have a better idea about what happens to UV textures before I would (as a programmer) load them from disk and jam them into graphics memory :P

The image is a visual test running in the emulator, and it's pretty much how I want this to look (sans botched texture coordinates of course). Pixelated character sprites in a 3D environment. Not only is it a good look, but the sprites compliment the low-res, unfiltered textures you get on the DS.

My goal is to crank out a very short game (20-30 minutes of gameplay) in about a month's time. This is a kind of distilled and truncated version of a larger game idea I have, but doing this will (1) get me out of creative stagnation and (2) give me a nice way to see how well that game would work.

...and perhaps get me to write LJ entries more often.

So Ubisoft has announced that they are keeping the Prince of Persia epilogue off the PC version. It actually surprised me how angry that made me. Perhaps it will surprise Ubisoft as well. At any rate, that's the last time I give them money for a PC game. I paid full price too... damn.