A slickproductions.org /forum/index.php?PHPSESSID=5f0fck550j2m4m2fpbtkj2vkm1&action=printpage;topic=2430.0 e:/My Web Sites/Slick Productions - FFIV Message Board/slickproductions.org/forum/index4cd9.html slickproductions.org /forum/index.php?PHPSESSID=5f0fck550j2m4m2fpbtkj2vkm1&topic=2430.0 e:/My Web Sites/Slick Productions - FFIV Message Board/slickproductions.org/forum/index4cd9.html.z x V-h^ Mt OK text/html ISO-8859-1 gzip @ Mt Wed, 11 Mar 2020 00:30:39 GMT 0 0 P V-h^ ! Mt
Print Page - New Tool: Visual SAK
Board of Slick
Banon's Donkey Farm => Game Modification Station => Topic started by: Squall on February 23, 2018, 08:37:49 AM
Title: New Tool: Visual SAK
Post by: Squall on February 23, 2018, 08:37:49 AM
Visual SAK: Visual Swiss Army Knife
A new tool developed that will be more like the real Swiss Army Knife - a toolbox in one package, easy to use, does few things but do them with precision! The toolbox will be focused on ripping of visuals from different ROM - Sprites, Graphics, Palettes, Tiles, ... but not casual ripping rather - finding, documenting, exporting and maybe in the feature importing.
Proof of concept: (http://www.bwass.org/bucket/Visual SAK - 1.png)(http://www.bwass.org/bucket/Visual SAK - 2.png)(http://www.bwass.org/bucket/Visual SAK - 3.png) (http://www.bwass.org/bucket/VisualSAK%20-%20screen5%20-%208bpp%20Mode3.png)(http://www.bwass.org/bucket/VisualSAK - screen6.png) There are plenty of things to be done - export of images, more operations on palette, project list, more formats, but I'm eager for some input before going on :D
Download: Version 3.1 (https://drive.google.com/file/d/1uvIxOWqOAwWzU-u_tdQrKCYzL9z52EdZ/view?usp=sharing)
P.S. Please download and share some feedback. Also if you find bugs, spell errors,... please report them here. If you have ideas how to make it better - please don't hesitate to suggest.
Title: Re: New Tool: Visual SAK
Post by: Tenkarider on February 23, 2018, 10:31:43 AM
sounds cool, what about compressed GFX?
Title: Re: New Tool: Visual SAK
Post by: Squall on February 23, 2018, 12:34:11 PM
Currently it supports GBA native LZ77-10 (also called LZ10 for short). The next compression that I will implement will be LZSS, the one used in FF5 SNES, but think the other FF titles use it too (need confirmation on that). I have plans to add compressed palettes too. Usually palettes are not compressed, but I know for sure some tiles do it.
P.S. Guys could you help me to speed up tests on next Template that I will implement: GBA 4 bit uncompressed GFX? I think FF6a keep most of GFX that way. What I need is: - at least 2 addresses in GBA ROM of GFX in 4bpp format uncompressed. - if you could provide their dimension that will be even better - Palette addresses of above. Must not be compressed :)
Title: Re: New Tool: Visual SAK
Post by: Madsiur on February 24, 2018, 10:36:14 PM
This is a really cool project! Which games outside FF4, FF5 and FF6 do you plan to support?
Here are some GFX locations for FF6A (E). I think most of them if not all are 4bpp uncompressed:
This is a really cool project! Which games outside FF4, FF5 and FF6 do you plan to support?
Thank you! Actually this tool is not connected with any game. I just use FFs as base to test functionality :D
Thank you very much for the info, Madsiur! It was helpful to test a new Template - GBA 4bpp (no compression). Unfortunately all the GFX require a tile-map, something that I have not implemented yet, so I couldn't see them properly.
I did notice that FF6a barely uses compression (at least not LZ10). Also all GFX uses a tile-map ... and tiles look scrambled inside ... Is there a particular reason for that, like weird copy protection of GFX? or just hmm bad programming :D
Anyway thanks to your data, version 2.0 is out! Download from first post.
P.S. Madsiur since FF6 uses LZSS variation do you know is it the same as FF5 LZSS one?
Title: Re: New Tool: Visual SAK
Post by: Madsiur on February 25, 2018, 06:22:44 PM
I did notice that FF6a barely uses compression (at least not LZ10).
It use compression for things that are not used often it seems like ending cut-scenes, some things like the Magitek Cart ride. Compressed GFX are labeled on the wiki ROM maps.
Also all GFX uses a tile-map ... and tiles look scrambled inside ... Is there a particular reason for that, like weird copy protection of GFX? or just hmm bad programming :D
Well a tilemap can be a form of compression. For monsters, it removed the possible empty tiles in the mold (32x32 or 64x64). As for something like the Window backgrounds, it's better to have 31 window tiles not ordered than having repetitions in a multiples of well aligned patterns. Tile order for sprites is also a way to prevent tile repetitions in sprite poses. However for something like portraits, I have no idea while tiles were put this way... Maybe reading them in batch of 4 (?) resulted in this being the optimal way to dispose them?
Madsiur since FF6 uses LZSS variation do you know is it the same as FF5 LZSS one?
That I have no idea, but it worth a try!
Title: Re: New Tool: Visual SAK
Post by: assassin on February 25, 2018, 06:33:45 PM
i dunno how many types of compression are in use. the only LZSS function i'm familiar with is C2/FF6D (but there are reportedly others): http://assassin17.brinkster.net/code2i.txt http://www.romhacking.net/utilities/1176/
something that came up in my search for the latter link was "Peer Sprite Viewer": http://geigercount.net/crypt/index.html
Title: Re: New Tool: Visual SAK
Post by: Squall on February 26, 2018, 03:02:22 AM
If the bit is 1: Read next byte from $D0 Write it on the circular buffer Write it on $D3 If the bit is 0: Read 16 bit value from $D0 That value contains an 'offset' and a 'length': O07 O06 O05 O04 O03 O02 O01 O00 | O10 O09 O08 L04 L03 L02 L01 L00 Do (length + 3) times Read a byte from the circular buffer with the given offset Write the byte on the circular buffer Write the byte on $D3 Increment offset (if offset > #$07FF then offset = #$0000) After consuming the 8 bits from the bitmap byte, a new bitmap byte is read from $D0. This is the FF5 LZSS algorithm. It uses $800 byte circular buffer. The repetition is 2 bytes - 11b Offset, 5b Length.
Thank you for your notes assassin! It seems to me its the same as your description. The only difference I see is that the Length (quantity as you call it) is placed in bits 3-7, while in FF5 its 0-4. Is there any chance you count bits from left to right? If so than its the same :D
Title: Re: New Tool: Visual SAK
Post by: Squall on February 27, 2018, 01:27:46 PM
Version 2.2 is out! - Added Template: SNES 3bpp (no compression) - Added Template: SNES 4bpp (no compression) - Major workflow change - Stability improvement
P.S. I need help to test 8bpp GBA graphics. If you know the addresses of graphic tiles of 8bpp in any GBA ROM (non-compressed or LZ77-10) please share some and the name of the ROM. If you happen to know their palette address that will be even better :thumbsup: Do you know games for GBA that use direct-color (non palette)? Do you know GBA games that internally store graphics in bpp different of 4/8? I wonder is it worth to implement other BPPs for GBA?
Title: Re: New Tool: Visual SAK
Post by: Squall on March 02, 2018, 08:43:05 AM
Short update: - I was able to implement LZSS version for FF6s. - Since now I have 3 methods of decompression, plenty ob BPP format, and number of ways tiles are laid in the memory, probably I will have to break improved Templates, so the user can choose separately BPP and Compression methods (otherwise it will be a long list to choose from). That was actually my initial idea, but that will require again major internal rework. That is the reason for no new release :(
On positive side: yes now I can decompress LZSS-FF6!!! assassin, thank you very much for your notes and provided links!!! They were invaluable in order to see the difference with FF5 and actually implement it. For those that are interested: - both FF5&FF6 uses LZSS with sliding window of $800 bytes (external), using 11b Offset and 5 b Length - small difference - the bits for Offset:Length pair is coded in different place in 2B (16b) - BIG difference - FF5 uses decompressed length as marker to end, while FF6 uses compressed length
Title: Re: New Tool: Visual SAK
Post by: Mauron on March 07, 2018, 05:32:44 PM
Playing with the compression menu, when I changed to GBA 4BPP LZ10 on something not in that format, I got both a friendly "Not valid LZ10 stream", and a more technical "Access violation at address 0047F09E in module 'VisualSAK.exe'. Read of address 00000006."
Title: Re: New Tool: Visual SAK
Post by: Squall on March 08, 2018, 05:26:54 AM
Thank you Mauron for reporting this. Could you tell me more about the setting where the bug appear? Probably a screenshot will be the best.
Title: Re: New Tool: Visual SAK
Post by: Mauron on March 10, 2018, 10:27:43 PM
Screenshots of both messages (https://imgur.com/a/JtJHb)
I purposely chose an invalid template for that data. It looks like it's not stopping after the first error message.
Title: Re: New Tool: Visual SAK
Post by: Squall on March 11, 2018, 01:36:17 PM
Screenshots of both messages (https://imgur.com/a/JtJHb)
I purposely chose an invalid template for that data. It looks like it's not stopping after the first error message.
Yeah no stopping ... I will work on that before releasing the new version. It seems I will have to separate compression from Template (the way it was originally designed) because the Template will grow exponentially with new bpp and compression methods added.
Thank you for finding that bug and reporting it, Mauron! Would you be so kind and share the impression from first time, no help/tutorial experience?
BTW you seem experienced in CT, do you know what compression method uses SNES version (it seems different from FF6)? I would love to implement it and add it to the tool :)
Title: Re: New Tool: Visual SAK
Post by: Mauron on March 11, 2018, 02:32:43 PM
Here's a document on Chrono Trigger's compression. (https://www.romhacking.net/documents/152/)
It seems like a good start. I would find it useful if I could load parts of a palette from different locations in a ROM, since Chrono Trigger handles weapon palettes that way, but I'm not sure how useful that would be overall.
Title: Re: New Tool: Visual SAK
Post by: Squall on March 11, 2018, 07:28:51 PM
O boy, why every new game should use own compressing method ...
Anyway from the document it seems CT uses again LZSS as base, but quite modified: instead of Control on next 8 bits (1 byte), it uses a packets - grouped Control up to 63 bits. Regardless it may sound better (63 bits > 8 bit) it actually adds extra bytes for packets overhead so I really doubt CT get better compression then FF5 & FF6 ...
Quote
I would find it useful if I could load parts of a palette from different locations in a ROM, since Chrono Trigger handles weapon palettes that way, but I'm not sure how useful that would be overall.
You can do it now, but its a little bit tricky. Keep in mind that regardless how many colors an image uses you can manually set how many colors to read for palette. Lets say the 16 color palette is composed by two locations: Addr1, Addr2. Addr1 has first 7 colors, Addr2 has last 9 colors: - read 16 colors from Addr2 so we get 9 colors to be last. You might have to use Addr2 - 7*2 as address for palette read!!! - read 7 colors from Addr1
Title: Re: New Tool: Visual SAK
Post by: Squall on March 27, 2018, 04:20:40 AM
Version 3.0 is out!!! - Internal engine rewritten! That change took me like 2-3 weeks, but is worth the time. Before I was using kind of Virtual Tile Engine. It was easy to use it to import things from game ROMS, but was slow. Now I use a 'double buffer', similar to what most of contemporary games do. Now drawing is FAST, no tear, no artifacts. That also allows me to easy export GFX - New feature: Export of Gfx (PNG only for now, other formats on demand) - Removed feature: Templates from 2.2. Now BPP is separated from compression (as intended in version 1). The main reason for this is that the list of Templates grows exponentially with increasing the supported BPP/compression formats. - Added new BPP: 2BPP (SNES) - Added new BPP: 8bpp (PC). PC yes, I was surprised that a SNES game would use it, but there are some like Wolfenstein 3D - Added new compression: LZSS-FF5, a modification of LZSS used in FF5 SNES - Added new compression: LZSS-FF6, a modification of LZSS used in FF6 SNES - Increased: The length of Names from 20 to 30 characters. Now you can be more descriptive :laugh:
*Download from the first post.
Title: Re: New Tool: Visual SAK
Post by: Squall on March 27, 2018, 10:31:02 AM
I want to keep previous post clean, that's why I'm starting a new post.
Proof of concept: (http://www.bwass.org/bucket/Visual SAK - 1.png) That picture demonstrate FF5 compression. I think its safe to say that no other tool can provide it :)
(http://www.bwass.org/bucket/Visual SAK - 2.png) Than is demonstration of FF6 compression. Could you help me with the correct palette for this image? I was trying to follow Madsiur's tutorial, but provided address wasn't giving the correct colors :(
(http://www.bwass.org/bucket/Visual SAK - 3.png) That is an interesting image. If somebody had told me that in a SNES game I will see a regular PC 8 bit image, I would probably laugh, but here it is :D
Title: Re: New Tool: Visual SAK
Post by: Madsiur on March 29, 2018, 05:35:29 PM
Than is demonstration of FF6 compression. Could you help me with the correct palette for this image? I was trying to follow Madsiur's tutorial, but provided address wasn't giving the correct colors.
This image can use multiple palettes and none will make all graphics look good. The one used in my tutorial is at $C20B91, starting with white ($FE7F so color $7FFE). This picture (https://www.ff6hacking.com/wiki/lib/exe/fetch.php?media=ff3:ff3us:tutorial:compressed:snespal1.png) show the 6th color being selected, hence your problem likely. This should make the shield and hand block look good at least. Good work btw!
Title: Re: New Tool: Visual SAK
Post by: Squall on March 30, 2018, 05:40:40 AM
Ahh, I was afraid it would use multi-palettes, but that is fine as far as the shield is correct :)
Thank you! Form what I have planned as compression, only Chrono Trigger SNES variant of LZSS is left for implementation. Thanks to Mauron's help at least I understand it, just need to code it :D As for BPP formats ... I'm doing experiments with non 8x8 tile formats. FF5 for example uses 8x12 for fonts. I'm sure many other games contain custom MxN format, so that will be interesting if I manage to pull it (you enter M, N, BPP and correct image appear).
After that I plan to focus more on the interface: - more operations with the Palette, like Load/Save/Reverse - more operation on the project list itself - hot keys, moving items, sorting,... - entering the addresses in an ASM friendly format. For example $C10000 instead of file-offset format $10000, to take in an account the presence of a header. - producing a document with a 'map' of found Gfx
Title: Re: New Tool: Visual SAK
Post by: Squall on April 18, 2018, 11:36:45 AM
Version 3.1 is out!!! - New feature: Import of a palette from zSnes savestate. - Added new BPP: 8BPP (SNES Mode 7). No planes. In mode 7 a tile takes 128 bites - because in the memory tiles are interwoven with the tile map - Added new BPP: 8bpp (SNES Mode 7 - II) same as above except it contain only the tiles (64 bytes each) - Added new BPP: 8bpp (SNES Mode 3) - each tile is 64 bytes, but each pixels is divided in 8 planes according to the SNES model of planes - Added new Compression: LZSS-CT, a modification of LZSS used in Chrono Trigger SNES
*Download from the first post.
Title: Re: New Tool: Visual SAK
Post by: Squall on April 18, 2018, 05:18:50 PM
Big thanks to Mauron for the help with CT compression!
Proof of concept: (http://www.bwass.org/bucket/VisualSAK - screen6.png) Finally, CT decompression is done. As I said earlier, its quite modified version of LZSS. It is interesting to me that CT uses compression a lot. Even palettes were compressed.
(http://www.bwass.org/bucket/VisualSAK%20-%20screen5%20-%208bpp%20Mode3.png) SNES has so many variations of 8bpp ... Mode 3 one seems most used for static images like title screen. So what is Mode 3 8bpp - a planar (8 planes) format, where planes have specific for SNES order in the memory.
Mode 3 also support two extra 8bpp models - Direct Color and index. If you know a game that uses some of these 2 models, please share. Even better will be if you can make a zSNES save state of it :)