|
Post by DarkKobold on Mar 20, 2019 16:27:47 GMT
Arkhan made a good point in the discord - compressing graphics, especially for level based games, should be fairly straight-forward.
The thing I don't know about is the representation of tiles in VRAM. I found this -
But this is over my head. I'm suggesting a potential collaboration, I'll write a C# program that loads in PNG, PCX, etc, and converts them to compressed ".bin" files. I just need someone's help with writing the decompression algorithm to load VRAM with either tiles or Sprites. I'm thinking Run-Length Encoding would be the easiest to implement. I've tested some of our files, and saved over half the space with just a simple RLE.
Any interest?
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Mar 20, 2019 20:00:13 GMT
RLE encoding is in fact, the easiest. You could go to NESDev for inspiration as the format of the files being compressed doesn't really matter unless you're doing a heavily customized algorithm ala Codemasters. There's a cool, public domain decompressor included in NES Screen Tool by shiru and also available here: github.com/AleffCorrea/BrickBreaker/blob/master/lib/rle.asm(skip all the way down to the "unrle" routine, that's the original one, the other is a dumb version used to write partial bursts of data).
In this case, the format goes like this: <tag> [data] <tag> <$00>, where data can be either <$xx> [more data] or <tag> <length> <$xx> [more data]. The tag is the byte that appears the least on the data being compressed.
unrle stx RLE_LOW sty RLE_HIGH ldy #0 jsr rle_byte sta RLE_TAG .1 jsr rle_byte cmp RLE_TAG beq .2 sta PPU_DATA sta RLE_BYTE bne .1 .2 jsr rle_byte cmp #0 beq .4 tax lda RLE_BYTE .3 sta PPU_DATA dex bne .3 beq .1 .4 rts
rle_byte lda [RLE_LOW],y inc RLE_LOW bne .1 inc RLE_HIGH .1 rts
PPU_DATA is $2007 on the NES, and basically the equivalent of $0002/$0003 on the PC Engine, so you can simply RLE compress data as 16 bit values in the first place or do something to alternate between $0002 and $0003 to adapt that algorithm to write to a 16-bit port.
|
|
|
Post by elmer on Mar 20, 2019 23:40:15 GMT
IMHO, this is a pretty basic piece of knowledge that a PCE programmer really should understand. Perhaps the official VDC docs would make it a bit clearer for you. If you're still confused, then I recommend that you play around in mednafen's debugger with some game's title screen (or the System Card's boot screen), and try changing the values in VRAM and see what happens. You should figure it out pretty quickly. Arkhan made a good point in the discord - compressing graphics, especially for level based games, should be fairly straight-forward. Yep. I'm glad that you're getting ready to push beyond the confines of what HuC provides and to start thinking about creating your own tools and low-level routines. Congratulations! You could go to NESDev for inspiration as the format of the files being compressed doesn't really matter unless you're doing a heavily customized algorithm ala Codemasters. Thanks for pointing out that resource!
|
|
|
Post by DarkKobold on Mar 21, 2019 0:35:53 GMT
IMHO, this is a pretty basic piece of knowledge that a PCE programmer really should understand. Perhaps the official VDC docs would make it a bit clearer for you. If you're still confused, then I recommend that you play around in mednafen's debugger with some game's title screen (or the System Card's boot screen), and try changing the values in VRAM and see what happens. You should figure it out pretty quickly.
So, I'd like to say, Punch and Arkhan really helped walk me through this on Discord as to how the hell it all works. It's really unintuitive and overwhelming at first.
I do want to say, the nice thing about HuC is that you don't need to know this stuff! When I say #incspr, it handles all the weird transformation into the multiplane sprite stuff. The point in mentioning this is, I think we've created a great game up to this point without me understanding a lick of this. While I'd like to learn it, it might be discouraging to a new programmer, and it really isn't necessary to know, and yet still create competent games in HuC.
We've got steps for me to do now, so I have a first path forward, to see if I can even create my own version of #incspr from an #incbin. Then, I'll work on compression.
|
|
|
Post by ccovell on Mar 21, 2019 2:44:15 GMT
That doc you linked to explains it about as clearly as it can be done, since game systems are all about bitplanes, tile offsets, etc.
Here's a snippet of my BG/SPR tile description from my PCEmon code:
; For each BG tile: ; Tile num 000-$7FF are VRAM add $0000-$7FF0 ; row 0..7 are at VRAM address 0..7 ; Bitplane 0 is at vram 0..7 LSB. ; Bitplane 1 is at VRAM 0..7 MSB. ; Bitplane 2 is at vram 8..F LSB. ; Bitplane 3 is at vram 8..F MSB. ; ; so... ; Get tilemap, read tile palette, store it *$10 ; read tile #, store it *$10 as address ; Read 2 bytes from address, add 8 to address, read 2 more bytes. ; Done for 8 horizontal pixels!
; For each Sprite tile: ; Tile num 000-$7FF are VRAM add $0000-$7FF0 ; row 0..F are at VRAM address 0..F ; Bitplane 0 is at vram 00..0F MSB,LSB (left 8 pixels,right 8 pixels) ; Bitplane 1 is at VRAM 10..1F MSB,LSB ; Bitplane 2 is at vram 20..2F MSB,LSB ; Bitplane 3 is at vram 30..3F MSB,LSB ; ; so... ; read tile #, store it *$40 as address ; Get what PCX line we want 0..F, add to address ; Read 2 bytes from address, add $10 to address, repeat 3 more times. ; Done for 16 horizontal pixels!
|
|
touko
Punkic Cyborg
Posts: 106
|
Post by touko on Mar 26, 2019 12:55:21 GMT
You have the LZ4 compression if you want : www.brutaldeluxe.fr/products/crossdevtools/lz4/index.htmlit's way faster than RLE, and the compression rate is better too . You have only to do the decompression,if fact the code is done but in 65816 assembly . LZ4_Unpack STA LZ4_Literal_3+1 ; Uncompress a LZ4 Packed Data buffer (64 KB max)
SEP #$20 ; A = Bank Src,Bank Dst
STA LZ4_Match_5+1 ; X = Header Size = 1st Packed Byte offset
STA LZ4_Match_5+2 ; Y = Pack Data Size
XBA ; => Return in A the length of unpacked Data
STA LZ4_ReadToken+3
STA LZ4_Match_1+3
STA LZ4_GetLength_1+3
REP #$30
STY LZ4_Limit+1
*--
LDY #$0000 ; Init Target unpacked Data offset
LZ4_ReadToken LDAL $AA0000,X ; Read Token Byte
INX
STA LZ4_Match_2+1
*----------------
LZ4_Literal AND #$00F0 ; >>> Process Literal Bytes <<<
BEQ LZ4_Limit ; No Literal
CMP #$00F0
BNE LZ4_Literal_1
JSR LZ4_GetLengthLit ; Compute Literal Length with next bytes
BRA LZ4_Literal_2
LZ4_Literal_1 LSR ; Literal Length use the 4 bit
LSR
LSR
LSR
*--
LZ4_Literal_2 DEC ; Copy A+1 Bytes
LZ4_Literal_3 MVN $AA,$BB ; Copy Literal Bytes from packed data buffer
PHK ; X and Y are auto incremented
PLB
*----------------
LZ4_Limit CPX #$AAAA ; End Of Packed Data buffer ?
BEQ LZ4_End
*----------------
LZ4_Match TYA ; >>> Process Match Bytes <<<
SEC
LZ4_Match_1 SBCL $AA0000,X ; Match Offset
INX
INX
STA LZ4_Match_4+1
*--
LZ4_Match_2 LDA #$0000 ; Current Token Value
AND #$000F
CMP #$000F
BNE LZ4_Match_3
JSR LZ4_GetLengthMat ; Compute Match Length with next bytes
LZ4_Match_3 CLC
ADC #$0003 ; Minimum Match Length is 4 (-1 for the MVN)
*--
PHX
LZ4_Match_4 LDX #$AAAA ; Match Byte Offset
LZ4_Match_5 MVN $BB,$BB ; Copy Match Bytes from unpacked data buffer
PHK ; X and Y are auto incremented
PLB
PLX
*----------------
BRA LZ4_ReadToken
*----------------
LZ4_GetLengthLit LDA #$000F ; Compute Variable Length (Literal or Match)
LZ4_GetLengthMat STA LZ4_GetLength_2+1
LZ4_GetLength_1 LDAL $AA0000,X ; Read Length Byte
INX
AND #$00FF
CMP #$00FF
BNE LZ4_GetLength_3
CLC
LZ4_GetLength_2 ADC #$000F
STA LZ4_GetLength_2+1
BRA LZ4_GetLength_1
LZ4_GetLength_3 ADC LZ4_GetLength_2+1
RTS
*----------------
LZ4_End TYA ; A = Length of Unpack Data
RTS
|
|
|
Post by DarkKobold on Mar 26, 2019 21:46:49 GMT
While this is awesome, my problem is I'm not proficient enough to wrap it into a C function. I don't even know how you'd begin.
|
|
|
Post by elmer on Mar 26, 2019 22:23:49 GMT
While this is awesome, my problem is I'm not proficient enough to wrap it into a C function. I don't even know how you'd begin. You'd want it in assembly, not HuC, or it would run horribly slow. And anyway ... since your game is HuCard and not CD, and you're writing in HuC, you almost-certainly don't have the free RAM to waste for the LZ window buffer anyway. It's probably best for you to just stick with one of the RLE variations for now, if that compresses your data well-enough for your needs.
|
|
|
Post by DarkKobold on Mar 26, 2019 22:37:46 GMT
While this is awesome, my problem is I'm not proficient enough to wrap it into a C function. I don't even know how you'd begin. You'd want it in assembly, not HuC, or it would run horribly slow. And anyway ... since your game is HuCard and not CD, and you're writing in HuC, you almost-certainly don't have the free RAM to waste for the LZ window buffer anyway. It's probably best for you to just stick with one of the RLE variations for now, if that compresses your data well-enough for your needs. After looking at a 16x16 sprite, it really doen't look like the VRAM version of sprites will compress well using RLE. RLE only really works because you're more likely to have 4+ identical colors in a row. When you convert it to the multiplane format VRAM, you are losing that advantage, especially if the sprite uses the entire palette. So, either I write a palette-based to mutliplane converter in ASM (which I'm incapable of doing), or compression really does nothing for me.
I think this is probably just beyond my skill level.
|
|
|
Post by gredler on Mar 27, 2019 0:02:15 GMT
Hearing the 4+ identical colors in a row makes me wonder if I could've made the art in a manner that would compress better? I do like to dither and manually alias the lines etc
|
|
|
Post by elmer on Mar 27, 2019 1:00:03 GMT
After looking at a 16x16 sprite, it really doen't look like the VRAM version of sprites will compress well using RLE. Yep, I was rather surprised when you said that you were getting 50% compression on graphics data using RLE. Hearing the 4+ identical colors in a row makes me wonder if I could've made the art in a manner that would compress better? I do like to dither and manually alias the lines etc Yikes ... please don't think like that! It's up to the artist to make stuff that looks great within the limits, and that should be your only concern. Then it's the programmer's job to make it actually work in the game. Are you guys running out of HuCard space? Is that where this is all coming from?
|
|
|
Post by gredler on Mar 27, 2019 1:07:50 GMT
Every since your Pro Motion implementation I've had so much more progress not having to spend the time doing palette cleanup, image splicing, and mappy. Because I am spending 90% less time on those menial tasks I am rapidly adding to ROM size by adding a lot of unique art, and I think DK is getting concerned that we will indeed run out of HuCard space if we keep up this pace and amount of content I am pushing towards him. I would really like to show you how far it's come along graphically thanks to your HuC branch. We are so rapidly progressing on the art side that I may soon have time to make some working examples in deflemask, since we've yet to receive anything from any of our musician friends to provide you with the missing links. DK doesn't think I can be our musician, but I think I can at least provide something that would be better than nothing!
|
|
|
Post by DarkKobold on Mar 27, 2019 16:29:01 GMT
After looking at a 16x16 sprite, it really doen't look like the VRAM version of sprites will compress well using RLE. Yep, I was rather surprised when you said that you were getting 50% compression on graphics data using RLE.
Yeah, RLE is perfect for palette-based graphics. I could easily write C code to decompress stuff into vram, it's just going to be... slow as molasses. Even if someone help me write in ASM, it might not be worth the 5 second load time in a HuCard game. I dunno.
|
|
|
Post by DarkKobold on Mar 27, 2019 16:44:14 GMT
Hearing the 4+ identical colors in a row makes me wonder if I could've made the art in a manner that would compress better? I do like to dither and manually alias the lines etc Yikes ... please don't think like that! It's up to the artist to make stuff that looks great within the limits, and that should be your only concern. Then it's the programmer's job to make it actually work in the game. Are you guys running out of HuCard space? Is that where this is all coming from? God no! As Elmer said, the quality of the graphics comes first. Especially since we don't even have a compression scheme in place, drawing around them would just be silly.
That said, the motivation is because we are at 600kB, and we don't have a single SFX or Husak song in place. We don't know if that's going to take 10kB or 300kB.... And even if it doesn't take much, we're still burning through ROM space.
So, I'm back to being a ROM Nazi. Gredler and I both have lots of ideas of little touches and enhancements we want to add... but those take ROM space. Code is cheap, Graphics are not. So, every new bird animation, or cat look, or blinking face on the title screen takes space from another piece of artwork. Additionally, my initial game design really painted us into a hole. We really have 4 games, and are working on 6 total, all combined into one. These games are animation heavy, and thus a lot of work for Gredler. This also eats away at available ROM space quickly.
For example, we have a creeping cat animation, 32x16 per frame. It's on screen for a short time, but gredler would love to make it 4 frames of animation. That's a high extra cost for a short cutscene.
If I could compress some of the more egregious sprites, like the big dog, the title screen circles, etc, that would open up so many more possibilities for adding more flair to the game, and make me less of a ROM Nazi.
I should note, there's also a lot of flair I want to add... this isn't just me telling Gredler no.
EDIT: I should also note, before I got into the weeds of how VRAM works, this seemed like it was going to be super easy. Compress graphics in MATLAB/C#, Uncompress in ASM. Turned out not to be the case...
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Mar 27, 2019 17:53:03 GMT
It might not be worth the 5 second load time in a HuCard game. I dunno. I think you might be overestimating the time needed to decompress graphics with RLE
|
|