|
|
Post by turboxray on Nov 4, 2019 15:59:06 GMT
I've been wanting to get this utility to a state that I can release it. Tentatively named pic2pce, cause I'm sick of coming up with cleverish names haha. I mean, it needs a name because it's more than just what the original utility was.  I've showed a couple of shots of it some years back. Here's a recent one. I just added undo/redo over the weekend, because you can actually draw on the tiles and change palette colors (as well as copy sub palettes from one to another). I have quite a bit of features to add, but I'll release it before adding them. The older "file open dialog" 32bit code for mac is no longer an option, so I'll only release the windows build for now. The algorithm for consolidating the smallest palette size for a set of tiles is pretty old (2006). And this was my first attempt at making my own widget framework (2012). So the source is a little embarrassing. I.e. won't be releasing it in its current state haha. But I'll provide binaries for windows, mac, and linux (ubuntu). Honestly, I probably could have just used Javafx and link my C code dynamically, and it would have been easier, but I like this kinda stuff. And in some ways, it's also easier because I can build whatever functionality I want fairly quickly with my own widgets. Looking at the pic, it doesn't look like much, but it's pretty powerful. There's also support for 3bit, 2bit, and 1bit tiles (cause I was making graphics for realtime compositing in demos, like the Thunder Force 4 transparency demo). It's not a lossy tool. It's meant for a pixel artist, or programmer, to have maximum control over image attributes. I didn't provide lossy options, simply because you can use Pro Motion and just take a screen snap of that and load it into this app. 90% of the time it will actually make a smaller subpalette set with PM's output than what PM did. Same for the java one for PCE that came out a few years ago. I'd often use PM as a first round draft, but then replace tilesets that had noticeable artifacts since I had more subpalettes to work with. It also outputs a image and a 'palette' image, so that you can make edits directly in Photoshop. Fragmare and I did this for Street Fighter 2 background upgrades. On that note, it also outputs a special embedded PCE color output, where R,G,B values in PCE range have an embedded code (offset by 1 or 2 values). This allows you to grab a tileset, have this app build out the smallest associated subpalette set, then take that special embedded bitmap and use it as a tileset in whatever favorite tile editor. You can write your own, but I also have a small commandline utility that'll build a tileset with extracted embedded subpalette info. That's more of an edge case scenario, but it allows you to build out a map in Photoshop as a large image, and just easily convert it so a PCE tileset, BAT, and subpalette set. Not gonna go into details, but there's a couple advantages of using PS for doing tilemaps over some tile map editors. So anway, taking the experience from building a widget framework, I had plans to make a nice PCE tilemap app. Basically stuff the other tilemap editors lacked for PCE specific stuff. I'll be working on it in tandem with this app. |
|
|
|
Post by dshadoff on Nov 4, 2019 16:34:42 GMT
Wow, looking forward to this !!
|
|
|
|
Post by gredler on Nov 4, 2019 17:22:12 GMT
Awesome, thanks for sharing!
|
|
|
|
Post by Arkhan on Nov 4, 2019 20:04:14 GMT
I swear I used some version of this years ago to convert high color image. Or was that something else that you'd released.
|
|
|
|
Post by elmer on Nov 4, 2019 22:01:58 GMT
I've been wanting to get this utility to a state that I can release it. Tentatively named pic2pce, cause I'm sick of coming up with cleverish names haha. I mean, it needs a name because it's more than just what the original utility was. Super cool stuff, excellent!   And this was my first attempt at making my own widget framework (2012). So the source is a little embarrassing. I.e. won't be releasing it in its current state haha. But I'll provide binaries for windows, mac, and linux (ubuntu). Honestly, I probably could have just used Javafx and link my C code dynamically, and it would have been easier, but I like this kinda stuff. Yeah, there's a bunch of my old source that just isn't in any state to release, too.  As for widgets, have you looked at wxWidgets or Qt ... both are tried-and-trusted ways of writing code that works on all of the different GUI environments. That's more of an edge case scenario, but it allows you to build out a map in Photoshop as a large image, and just easily convert it so a PCE tileset, BAT, and subpalette set. Not gonna go into details, but there's a couple advantages of using PS for doing tilemaps over some tile map editors. Yep, building a large map as a huge bitmaped image, rather than as a tileset, has a number of advantages, and takes you into the mid-to-late 1990s and GBA-era of how professional developers actually made their art. But you don't normally go directly to a pure BAT, that's far too wasteful of your limited console memory ... there's an intermediate step, a "block" that is basically a 2x2 mini-BAT. That makes your map 1/8 the size of a pure BAT (or 1/4 if you use more than 256 blocks ... in which case you can usually spare a couple of bits for collision info or tagging). You can see the technique used by people like Falcom in the LoX games, by Hudson/MediaWorks in Seiya Monogatari, and almost-certainly in lots of other games, too. I was/am tempted to write up something, complete with art examples, and possibly some HuC6280 code for the map drawing, but, like so many things, it's still on the woulda/coulda/shoulda list of things-not-done.  Anyway, it's easy to use something like ProMotion or grafx2 (or originally DPaint/DAnim) to make maps in that way, and then call an external command-line tool to process the bitmap into your desired tiles/blocks/map/palette format and, at the same time if you wish, spit that out to a connected console for viewing on a TV. ProMotion in particular is set up to accomodate exactly that kind of development setup, because professional developers asked for it. These days, with the fast USB connection on the Turbo Everdrive, it would be fairly-easy to recreate that kind of setup, and even do "live" updates to a running game if you wished ... or you could just auto-launch an emulator, which would be another way of doing it. |
|
|
|
Post by gredler on Nov 4, 2019 22:14:27 GMT
ProMotion in particular is set up to accomodate exactly that kind of development setup, because professional developers asked for it. That next level tilemap optimization sounds great, but again the artist over needs a bit more explanation. Would that let us (artists) use more than the 16 palettes and 256 tiles, or would this be performance improvements? These days, with the fast USB connection on the Turbo Everdrive, it would be fairly-easy to recreate that kind of setup, and even do "live" updates to a running game if you wished ... or you could just auto-launch an emulator, which would be another way of doing it. I have been using this process for a while now, both the emulator and launch to TED, but not "live" updates/real time. I make changes, save art, build&run the game, see my changes. It's very fast, but would be nice to see things update faster especially when I get into more animations via palette cycling. |
|
|
|
Post by turboxray on Nov 5, 2019 15:40:09 GMT
I swear I used some version of this years ago to convert high color image. Or was that something else that you'd released. I think really early on, I had a few people try out an upgraded command-line version (my first SDL project). It just showed the two pics (original and conversion), and you could use the arrow keys to choose the base color to build from. I know Nodt used it at some point for MSR (it had a BG #0 bug in the palettes, but I guess he somehow manually fixed the output). But yeah, potentially that version haha As for widgets, have you looked at wxWidgets or Qt ... both are tried-and-trusted ways of writing code that works on all of the different GUI environments. I had originally looked at them. But I wanted to make my own as well so this gave me that excuse. But you don't normally go directly to a pure BAT, that's far too wasteful of your limited console memory ... there's an intermediate step, a "block" that is basically a 2x2 mini-BAT. That makes your map 1/8 the size of a pure BAT (or 1/4 if you use more than 256 blocks ... in which case you can usually spare a couple of bits for collision info or tagging). Ohh, I just meant as an tilemap editor. I made tilemaps, collision maps, etc in Photoshop. Just made a small cmd line utility to convert the different layers into whatever (BAT+tileset, collision maps, etc). Having the tileset run through this pic2pce, the bmp output would have the embedded differences in the RGB values. So when the bmp layer ran through the conversion util, it would extract out which subpalette that tile belonged to (as long as you didn't modify the tileblock and copied along 8x8 boundaries) and build the BAT from there. As far as the Tile map app, I've always wanted the ability to do dynamic tiles (pieces of the tileset than can be swapped in and out as you scroll around) so that you could have a larger tileset than you would have allocated in VRAM. I remember doing some logistics for Xymatia (sp?) where there wasn't enough tiles to all fit in vram along with sprites, but since it was auto-scrolling I had gotten the idea to simply stream the 32 tiles across a few frames. That sound it's more in line with what you're talking about? It was basically a bitmap + small size BAT. And loop back onto itself. It also allowed for tile flipping (pretty easy to do in real time). It was either do that, or a dynamic tileset approach but that needed some specialized utility support to figure that out. |
|
|
|
Post by elmer on Nov 6, 2019 23:21:31 GMT
I had originally looked at them. But I wanted to make my own as well so this gave me that excuse. Hahaha ... yep, I get it, as a programmer, you wanted to have the fun of learning how to do something new.  Ohh, I just meant as an tilemap editor. I made tilemaps, collision maps, etc in Photoshop. Just made a small cmd line utility to convert the different layers into whatever (BAT+tileset, collision maps, etc). Having the tileset run through this pic2pce, the bmp output would have the embedded differences in the RGB values. So when the bmp layer ran through the conversion util, it would extract out which subpalette that tile belonged to (as long as you didn't modify the tileblock and copied along 8x8 boundaries) and build the BAT from there. Yep, I thought that you were talking about Map-making ... so was I! It is a pretty neat idea to use the low-bits of the RGB to tag palette-indices so that you can fix things again up after Photoshop rearranges your palette entries ... but it just makes far more sense to me to use an art tool that doesn't frak around with your palette, since there are plenty of them. As far as the Tile map app, I've always wanted the ability to do dynamic tiles (pieces of the tileset than can be swapped in and out as you scroll around) so that you could have a larger tileset than you would have allocated in VRAM. I remember doing some logistics for Xymatia (sp?) where there wasn't enough tiles to all fit in vram along with sprites, but since it was auto-scrolling I had gotten the idea to simply stream the 32 tiles across a few frames. That sound it's more in line with what you're talking about? It was basically a bitmap + small size BAT. And loop back onto itself. It also allowed for tile flipping (pretty easy to do in real time). It was either do that, or a dynamic tileset approach but that needed some specialized utility support to figure that out. I'm not really talking about any of those things, but they're all possible (and fun-to-program) things that you can do when you're an assembly-language programmer, you're willing to write your own conversion utilities, and you've got an artist who will work closely with you to push the limits of whatever system you come up with. That next level tilemap optimization sounds great, but again the artist over needs a bit more explanation. Would that let us (artists) use more than the 16 palettes and 256 tiles, or would this be performance improvements? It's a compromise between using something like the mapping technique that is currently in HuC (that most people from the Amiga/ST era used when we first started developing for consoles) ... and the full-freedom-but-high-memory-usage of storing maps in BAT format. I'll start up another thread, rather than continuing to derail this one about Tom's lovely-looking editor. |
|
|
|
Post by turboxray on Nov 7, 2019 17:19:09 GMT
One of the parts of this project that never got around to finishing, was support for 8x4, 8x2, and 8x1 tiles (mostly for still images). It'd supply the extended BAT for it, and you'd just need to have an H-int routine to do the updating for it.
About the consolidation algorithm. It's greedy in that it takes the pass of all consolidation attempts, and chooses the one that results in the best output. I.e. It n^2, as it attempts to consolidate by taking current n and consolidating with k -> then rematches all tiles this new subpalette configuration. If there is multiples with the same best values, it will build out their configurations one more level - but if those result in more 'ties', then it just picks the first one. I had plans to set this level by the user, but for now it's 1. It does this until nothing else can be consolidated. On some pics it can take up to 30+ seconds, but most are within a second or two. It's not a very smart algorithm haha, but it's pretty effective.
What it doesn't current do, but I still plan to do, is have an option to bring in a palette separately and then have a tileset matched against that.
|
|
|
|
Post by Arkhan on Nov 10, 2019 2:50:09 GMT
Yeah Tom, that's the one I am thinking of. I remember loading and messing with a bunch of VGA images like Doom/Heretic title screens and crap.
Man that was awhile ago lol
|
|
|
|
Post by turboxray on Nov 19, 2019 8:14:45 GMT
*Hopefully I tagged this youtube video right..
Just showing off the app. I've spent a lot of time on it over the past two weeks, mostly converting the mouse/click code over to the new widget framework. I didn't have a save 'project' option before (just BMPs), soI added that (save and load a project). The widgets can be individually enabled to directly set the undo flag based on whatever action. I made a 'transform' callback member type. It'll take a number of args pass as void*. You just need to make the transform func yourself and handle casting the arguments back. I mostly did this for x/y coords, but after I afterwards I realized the callback function for the widget action state should just do it itself. What else..
Subpalettes can be copied to either a free pal space or over an existing one (left-click shows the yellow source selection border, right-click shows the destination border. A second right-click on the same subpalette initiates the copy). In the tile selection window, the associated palette # is shown, but can be changed (cycle through), and then assigned with the ">" button. Same for colors inside a subpalette; left-click selects the source color, right-click overwrites the destination color entry. You can modify the RGB elements directly from a selected color. Right-clicking on a pixel selects the corresponding subpalette color, and updates the RGB to show its values. Clicking on a tile outside of the pink border tile, selects it (i.e. scrolls by one tile - 8 way diagonal scrolling).
I made some new widgets; toggle for turning something on/off. A cycle-through toggle with any number of states (the 4bit button for example). I just created the radio group button(s) tonight (that's the Pal Sel, Pal Only, etc). Left clicking on the original image chooses a tile (holding down the mouse button allows you to see update in realtime). Right clicking on the same image changes the radio group; you can do both - holding down the left mouse button to auto update as you move the mouse over the image, and right clicking to toggle the PCE image view). I added support to the widgets in which the the state can be directly modified, and the widget member function update() can be called to redraw itself. I think I have about 5 different widgets, and some do multiple duty (drag/draw/etc).
The undo support is currently limited to 32 entries. I need to increase that and also make it a circular buffer (a current TODO). There is a mask support to keep track what was modified, so that at a later point you could click a drop down menu and directly click an undo state, or just see a history of your actions.
Some of the copy functions for tile -> palette reassignments or pal->pal copy, only reassign matching colors - the rest get assigned to color #0. I have plans to do a three-point distance search against RGB and one against just Y (from YUV) to get a weighted nearest neighbor assignment. That's important TODO.
The 'error' view isn't hooked up yet, but that should be done soon.
Anyway, I spent most of my time on the widget stuff because now it's pretty easy to add functionality. If you're wondering why it looks like text? Well... I wrote my own print function and font, so it's faster to call a print routine than sitting there drawing buttons and what not hahah. Next on the list of making a group widget which does nothing more than act as a relocatable form (widgets inside it use relative coords rather than main window coords).
|
|
|
|
Post by gredler on Nov 19, 2019 19:37:09 GMT
Cool stuff dude thanks for sharing the video, and I'm looking forward to hearing more! Is this available to try anywhere, or is it still private - and if it is are you planning on releasing this?
I would be super curious to pass some of our assets through it to see what they do, or what potential savings & improvement impacts it would have.
Love seeing new tools and workflows available to the content creators - thanks again!
|
|
|
|
Post by turboxray on Nov 19, 2019 20:40:56 GMT
Cool stuff dude thanks for sharing the video, and I'm looking forward to hearing more! Is this available to try anywhere, or is it still private - and if it is are you planning on releasing this? I would be super curious to pass some of our assets through it to see what they do, or what potential savings & improvement impacts it would have. Love seeing new tools and workflows available to the content creators - thanks again! It's only private right now pretty much because I tend to use it and photoshop together (going back and forth between the two), but I want to add a few more features so that it can be much more self contained. But yeah, definitely plan on releasing it soon. I mean, I've done a lot of work with this tool as is, so it's definitely valuable to me in its current state but I want to make it more flexible. It's fairly close to a release for testing. It just needs a few more tile/palette editing/tooling options. |
|
|
|
Post by turboxray on Nov 21, 2019 3:00:26 GMT
For those that rips graphics from games/assets/pics, or just convert images to PCE format, what functionality would you like to see? I will be adding lossy conversion code (for color reduction), but that isn't the main purpose (a one and done click to convert, but more of an aid). I have a few ideas that are better then just euclidean distance "nearest neighbor" reduction method. But I referring to drawing tools, statistics (per tile, per palette, per tileset associated to a palette, overall image, etc), more tile editing tools?
|
|
|
|
Post by turboxray on Nov 25, 2019 17:51:53 GMT
I originally wasn't going to have any lossy options in the app, because it was supposed to be a tool for manual precision conversion (for whatever that means haha). But I figured what the hell: Old: new:
Old: new:
It's nothing special (just Euclidean distance in RGB space), but it works fairly well. The reason I implemented this for the initial conversion phase, is that I wanted the support for when you want to manually override a tile; re-copy a source tile to destination tile but chose your palette manually and have it nearest match against that pal color set. I'm surprised by how efficient it works, considering how it chooses the best sub palette to match against is pretty dumb. So I'll do some more work on this. I'll have a couple other colors paces and the option to set a bias (per tile and whole image).
|
|
