|
Post by elmer on Nov 7, 2019 1:44:16 GMT
This may have already been talked about before, and be well-know to the artists and programmers here, but I've not seen it mentioned in the last few years, so here goes ... While early PC Engine games sometimes used 16x16 pixel bitmapped "tiles" to create their level maps, similar to the method that is currently built into HuC and PCEAS, later games often used a similar-but-more-flexible technique that both reduces memory usage, and can increase the graphic quality of the results. Here is one example from Falcom's Legend of Xanadu 2, which is often seen as one of the best-looking games on the PC Engine. In order to demonstrate the technique used, I'll employ Pro Motion NG as the application for converting this example level into a tilemap. In both of the LoX games, Falcom split their huge level maps into 1024x1024 pixel sections, blanking the screen and decompressing a new section into RAM/VRAM as you move from one section to the next. Here is the map for Seran Village, early on in the LoX2 ... When the image is converted to 16x16 pixel bitmapped tiles, in a similar way to the conversion in HuC, ProMotion generates 239 tiles. This results in a small MAP size of 4KB, but it uses nearly 30KB of VRAM for the tile data ... and anyway, it wouldn't actually look correct in HuC, because lots of the 16x16 tiles use more than a single sub-palette, which HuC cannot natively handle. When the image is converted to 8x8 pixel characters, ProMotion generates 492 characters, which only use 16KB of VRAM, half the memory size of the 16x16 tiles. When duplicate character definitions are removed (because ProMotion's tiles are 8-bits-per-pixel, rather than the PC Engine's 4-bits-per-pixel), then the usage comes down to 488 characters (i.e. 4 characters in the level map are used with different sub-palettes). Unfortunately, the MAP, which is now basically the same format as the PC Engine's BAT, has grown to a rather large 32KB. The middle ground, that corresponds to how Falcom actually stores their level maps, is to use an external converion utility, and to generate a set of three CHR/BLK/MAP files, where the CHR data is the small 16KB set of 488 characters, where the MAP is made up of single-byte indices into the BLK table (thus taking 4KB, just like HuC), and where the BLK table defines a 2x2 CHR (16x16 pixel) set of BAT data (i.e. the CHR number and the sub-palette number) which only takes up 2KB. The end result is that CHR/BLK/MAP data not only takes up less space than either of the other methods, it also provides for using multiple sub-palettes within a 16x16 area, and it combines the same character data from different blocks, which can make background-character-animations more efficient. You can clearly see that Falcom are using this scheme when you take a look at the character data that the game uploads to VRAM when it displays this level ... Another thing that is worth looking at is how Falcom does the background animation for the water on this level ... Here is a dump of the VRAM usage of the animation, shown twice, top-and-bottom, in the two different sub-palettes that are used for the lit-vs-shadowed water ... FYI, to make things easy to examine/test for yourself, you can download an archive of all of the image files here ... lox2_map_example.zip (157.41 KB)
|
|
|
Post by gredler on Nov 7, 2019 1:45:41 GMT
What a fantastic write up, thank you so much for doing this!
|
|
|
Post by ccovell on Nov 7, 2019 4:38:53 GMT
|
|
|
Post by gredler on Nov 7, 2019 5:15:29 GMT
This was mesmerizing, thank you so much for sharing it! Such professionals, with great planning, it's easy to see how their project turned out so masterfully. I wonder how they went about visualizing the tilemap mirror offset results when planning the level designs; and the method for picking hardmode tile offsets. Wow, amazing.
|
|
mooz
Deep Blooper
Posts: 29
|
Post by mooz on Nov 7, 2019 9:22:15 GMT
Cool. I'm currently working on a tilemap converter and the associated display routines for HuDK. I think I'll make those "mipmap" tilemap thingies default.
|
|
TailChao
Gun-headed
I Must Eat Muffin Gear.
Posts: 68
Fave PCE Game Overall: Bonk's Adventure
|
Post by TailChao on Nov 7, 2019 15:55:10 GMT
Morphcat's video does an excellent job at explaining the techniques used for Micro Mages' stages data, but the use of blocks or metatiles is extremely common. As a side effect, there's lots of reverse engineered commercial games to compare with. I recommend looking at Sonic the Hedgehog's Level Format and the accompanying Physics Guide, as both the visual and interaction tags for each part of the stage are conveniently rolled together.
|
|
|
Post by turboxray on Nov 7, 2019 16:55:13 GMT
Yeah, a lot of NES games do this - meta-meta tiles. They also tend to have maps as meta-screens too. I had completely forgotten that HuC did one subpalette per 16x16 metatile block. Using 30k for the whole 1024x1024px map in vram seems wasteful. Elmer, doesn't the game employ masking sprites for the BG too? Low layer priority but higher sprite priority to pop out sections of the BG dynamically over the sprite? Doesn't leave much room in vram; 32k for map, 16k for tiles, and the rest 16k for SATB, sprites, and mask sprites.
I'm guessing the mirrored tiles in vram were flipped on the fly when written to vram (saves space in main memory). Later gen PCE games were doing that.
|
|
|
Post by gredler on Nov 7, 2019 18:34:57 GMT
When we first started Catastrophy I did everything as 8x8 tiles, But we then ran into issues (It's been so long, I don't remember what they were), so DK asked me to use 16x16 tiles, so I then baked it down into 16x16 tiles. We've used 16x16 since then but man 8x8 would allow for so much more variety. If I ever work on another turbo project I would hope to do it in 8x8 tiles. The Mini Mages example of mirroring tiles for savings is not possible on turbob, correct?
|
|
|
Post by ccovell on Nov 8, 2019 1:07:33 GMT
The Mini Mages example of mirroring tiles for savings is not possible on turbob, correct?
Both the NES and TurboGrafx have the same limitations: BG tiles cannot be flipped in hardware, but sprite tiles can. Mirrored BG tiles have to be flipped by software or stored that way in ROM before being uploaded to VRAM; and the NES is the same, although that Micro Mages game uses VROM.
|
|
|
Post by gredler on Nov 8, 2019 1:27:01 GMT
Both the NES and TurboGrafx have the same limitations: BG tiles cannot be flipped in hardware, but sprite tiles can. Mirrored BG tiles have to be flipped by software or stored that way in ROM before being uploaded to VRAM; and the NES is the same, although that Micro Mages game uses VROM.
DarkKobold are we able to mirrior tiles in software and you've been holding out on me?! this could let us fit so much more ART!!!!
|
|
|
Post by DarkKobold on Nov 8, 2019 6:38:48 GMT
Both the NES and TurboGrafx have the same limitations: BG tiles cannot be flipped in hardware, but sprite tiles can. Mirrored BG tiles have to be flipped by software or stored that way in ROM before being uploaded to VRAM; and the NES is the same, although that Micro Mages game uses VROM.
DarkKobold are we able to mirrior tiles in software and you've been holding out on me?! this could let us fit so much more ART!!!!
"We" are able to do it, in the same way that "We" are able to write compression algorithms. It's possible, if I wrote some complex assembly above my head. Additionally, you'd still be limited to 256 tiles, just some tiles would be mirrors. So it'd save ROM space, be less useful than compression, and yeah...
The other way to get more art in the game is to switch platforms to the Nintendo Switch. Part of the fun of this is doing a game for an old system, which comes with limitations we are already pushing way too hard against.
|
|
|
Post by gredler on Nov 8, 2019 7:06:12 GMT
DarkKobold are we able to mirrior tiles in software and you've been holding out on me?! this could let us fit so much more ART!!!!
"We" are able to do it, in the same way that "We" are able to write compression algorithms. It's possible, if I wrote some complex assembly above my head. Additionally, you'd still be limited to 256 tiles, just some tiles would be mirrors. So it'd save ROM space, be less useful than compression, and yeah...
The other way to get more art in the game is to switch platforms to the Nintendo Switch. Part of the fun of this is doing a game for an old system, which comes with limitations we are already pushing way too hard against.
youtu.be/vCadcBR95oU
|
|
|
Post by elmer on Nov 8, 2019 21:17:55 GMT
Thanks Chris, that's an absolutely excellent video showing the tortuous lengths that developers had to go to back in the 1980s to cram stuff into limited computer memory or cartridge space! Hahaha ... dealing with everyone's different naming for things can be a bit of a PITA, especially when 2 people use the same name for different concepts. I'm old enough that 8x8 pixel thingies are "characters", and that the hardware that displays them is a "character generator" ... which is the convention that Hudson's PCE, Sega's Genesis, and Nintendo's SNES official docs use, too. I don't know when people started calling them "tiles". I always get confused when people talk about tiles because for me, if you're going to use that name at all, they're the 2x2 character 16x16 pixel areas ... i.e. the same as a "block aka BLK". Anyway, yep, as the video shows, you can meta-meta- everything up to whatever levels of repetition you need in order to fit your game into memory. The downside is that the backgrounds can start to visibly *look* repetitive when they're sharing large meta-meta-tile areas. From my POV, small 2nd level meta-tiles (say 32x32 or 64x64 pixel), followed by a 3rd or 4th set of meta- repetition/optimization was really an 8-bit-generation solution in order to get around the 64KB-or-less limited memory available for games. Once you get passed the days of the PCE's original 8KB of work RAM, or the paltry 64KB in the original PCE CD, then IIRC the rest of the 16-bit generation of games moved to using the single-level CHR/BLK/MAP (aka TILE/META-TILE/MAP), and then used data compression (such as LZW, LZSS, or all of the wonderfully inventive variants) in order to fit everything onto a cartridge, or into a single load from CD. You can see that in the "Sonic the Hedgehog" level documentation that TailChao kindly pointed out. Sure, you can call SCHG's 256x256 pixel areas "meta-meta-tiles" if you like, but personally, I'll stick to calling them a "MAP", and the overall 64x8 arrangement of them a "LEVEL" ... it's probably an age thing. Thanks TailChao , those give another lovely example of the technique as used in shipped game. Once you expand the MAP data from 1-byte-per-BLK to 2-bytes-per-BLK, then you can easily expand to using 512 or 1024 BLKs, and still have bits left for other data, such as collision ... in just the way the SCHG developers did. Or you can just stick to 256 BLKs, and use a small lookup table to get the collision info, in the way that Falcom did. There is no "right" way, it's all just a mater of design choices, and understanding the costs-vs-benefits for the particular type of game that you're writing. Using 30k for the whole 1024x1024px map in vram seems wasteful. Elmer, doesn't the game employ masking sprites for the BG too? Low layer priority but higher sprite priority to pop out sections of the BG dynamically over the sprite? Doesn't leave much room in vram; 32k for map, 16k for tiles, and the rest 16k for SATB, sprites, and mask sprites. Yes, the LoX games both use masking sprites in order to create a 2nd-layer effect. But I guess that I wasn't clear about the memory usage ... nope LoX absolutely does not upload the entire 32KB of a 1024x1024 pixel BAT to VRAM ... the section map stays in main RAM as a 4KB MAP and a 2KB BLK table. Because they're displaying a 240-wide screen, the VDC BAT in the LoX games is 32x32, taking just 2KB of VRAM, rather than the 64x32 (4KB) BAT that they would need to scroll a 256-wide screen. Just like in any other scrolling game, the BAT is updated in realtime as the player moves around the level. IMHO, Falcom were pretty efficient with their use of VRAM ... despite their choice to allocate VRAM memory space for both the Status Bar and the Dialog Box at the same time ... 0000-03FF 2KB BAT 0400-04FF 1KB SATB1 0500-05FF SATB2 0600-07FF 1KB Permanent Sprites 0800-0FFF 4KB Status Bar 1000-1FFF 8KB Dialog Box. 2000-2FFF 8KB BG Characters (level shared characters) 3000-3FFF 8KB BG Characters (section-specific characters) 4000-4FFF 8KB BG Characters (section-specific characters, usually includes dynamic animated tiles) 5000-57FF 4KB Mask sprites. 5800-7FFF 20KB Level sprites. The other thing to note is that they organized their game assets in 8KB chunks of data, decompressing 8KB chunks as-needed into a bank in memory, and then copying them to VRAM (if desired). Doing it that way is faster than just decompressing the data directly into VRAM, which requires a slow inner-loop because of the ring-buffer. If you look at the Seran Village charset, you'll see that the 1st 8KB (256 characters) is used for the basic ground foliage that is shared by a lot of the 1024x1024 sections in the level, then there is 8KB for the house graphics, and finally, 8KB for the dynamic characters for the water, with some unused characters in all three areas. That kind of organization is pretty easy to develop for, as long as you have some simple customized conversion tools and a decent container format for the compressed assets ... what I called a META_BLOCK when I was doing the translation. I threw together something similar for compressing the font data in TEOS, and put it on github ... aplpak. Anyway, IMHO, and just to be very clear, that's an O-for-a-personal-opinion-not-a-fact, Falcom's techniques and technology was far more advanced than Hudson's at the time. Seiya Monogatari's internals are a bit of a mess in comparison to LoX, particularly when you look at the tech-improvements in LoX2, which was only Falcom's second console/CD game.
|
|
pokun
Gun-headed
Posts: 85
Homebrew skills: HuC6280 assembly
|
Post by pokun on Nov 9, 2019 17:46:04 GMT
It doesn't help that the term tile sometimes also is used for the actual pattern data of both background characters and sprites, that's something that often confused me since I associated tiles with backgrounds. Hudson seems to have inherited the terms BG characters and sprites from the TMS9918 video chip. The term sprite seems to have stuck ever since and used in modern games (even if they are software sprites), but characters are often called tiles for some reason, even though pretty much all video hardware called them BG characters or something similar. Nintendo calls them characters or CHR but coined their own names for some other things. Sprites are objects or OBJ on all Nintendo systems that I know of and SAT/SATB is called OAM (Object Attribute Memory).
I guess the term character comes from the fact that it was initially for displaying text on computer screens (the technique of using characters is often called text mode) but it turned out to be very useful in games as well by redefining the characters into images of rocks, bricks ladders and stuff, and by adding colour attributes to them.
|
|
|
Post by elmer on Nov 10, 2019 23:24:28 GMT
I'm guessing the mirrored tiles in vram were flipped on the fly when written to vram (saves space in main memory). Later gen PCE games were doing that. Nope, Falcom don't do that in the LoX games, and AFAIK neither did Hudson/MediaWorks in Seiya Monogatari. Once you start doing detailed "lit & shadowed" backgrounds, the kind that marks the best-looking console art from the 16-bit generation, you tend to find that there's not a lot of X-flipped and Y-flipped characters in your art, and so it's not worth making special arrangements to deal with them on a platform like the PCE, which doesn't natively support them in hardware. IMHO one of the biggest design mistakes of the 16-bit era, was when the design engineer of the Megadrive's VDP actually chose to use 2-bits of the BAT (aka the PATTERN NAME TABLE) for the X & Y flip capability, dropping the Megadrive down to 4 possible palettes instead of the 16 palettes that the PCE supports for its backgrounds. Then again, the PCE was designed by a group of folks that developed games, and had the experience of having to work their way around the hardware-limitations of other machines, and the design engineer of the Megadrive's VDP has famously said that he never talked to any of the software guys while designing the chip. Here are another couple of example images. The second level of LoX2, the port town of Razan ... This level displays a lot of the characters in more than a single palette. The obvious example is the roof of the houses, which is shown in both red and blue. The other thing that stands out is that even though the houses share a lot of blocks/BLK/meta-tiles, every house is actually unique, just like the houses in Seran Village. That's something that really helps to give the level map a feeling of quality. Converting this map into CHR/BLK/MAP format results in the use of 409 characters. Enabling my converter's option to detect and remove both X-flipped and Y-flipped characters ... still results in the use of 409 characters. Not a single character in this section is flipped, they are all unique. Going back to Seran Village ... there are only 12 flipped characters in that out of 488, so 2.5% ... that's just not worth Falcom writing any special-case handling for. Switching to Seiya Monogatari, one of the other candidates for "best-looking game", and its Yalem Village map ... Well, first of all you can see that Seiya Monogatari is using actual sprites, rather than masks, to add details and a 2nd layer. Apart from that, the actual graphics don't look as good as Falcom's work IMHO, and the repetition of elements is much more obvious. For instance, there are a bunch of identical houses, and the ground texture is a single repeated block. So perhaps this would show more evidence of flipped characters? Well, actually no. Out of the 352 characters used, only 1 is flipped. Lastly, let's look at another part of Seiya Monogatari's Yalem Village, the Church interior (just a single screen's worth) ... Now this looks great, but when you see all of that repetition on the floors, surely there are a bunch of flipped characters in this screenshot! Well, yes, there are some, but once again, not many ... only 11 out of 157. So, from my POV, the conclusion is ... the better your art is, the less reason you'll find for bothering to deal with flipped characters on the PC Engine (or anywhere else, I'd expect). As before, to make things easy to examine/test for yourself, you can download an archive of all of the image files here ... lox2_map_example2.zip (142.93 KB)
|
|