Topic: New Tool: Visual SAK

[Squall]

Visual SAK: Visual Swiss Army Knife

A new tool developed that will be more like the real Swiss Army Knife - a toolbox in one package, easy to use, does few things but do them with precision!
The toolbox will be focused on ripping of visuals from different ROM - Sprites, Graphics, Palettes, Tiles, ... but not casual ripping rather - finding, documenting, exporting and maybe in the feature importing.

Proof of concept:


There are plenty of things to be done - export of images, more operations on palette, project list, more formats, but I'm eager for some input before going on :D

Download: Version 3.1

P.S. Please download and share some feedback. Also if you find bugs, spell errors,... please report them here. If you have ideas how to make it better - please don't hesitate to suggest.

[Tenkarider]

sounds cool, what about compressed GFX?

[Squall]

Currently it supports GBA native LZ77-10 (also called LZ10 for short).
The next compression that I will implement will be LZSS, the one used in FF5 SNES, but  think the other FF titles use it too (need confirmation on that).
I have plans to add compressed palettes too. Usually palettes are not compressed, but I know for sure some tiles do it.

P.S. Guys could you help me to speed up tests on next Template that I will implement: GBA 4 bit uncompressed GFX? I think FF6a keep most of GFX that way. What I need is:
 - at least 2 addresses in GBA ROM of GFX in 4bpp format uncompressed.
 - if you could provide their dimension that will be even better
 - Palette addresses of above. Must not be compressed :)

[Madsiur]

This is a really cool project! Which games outside FF4, FF5 and FF6 do you plan to support?

Here are some GFX locations for FF6A (E). I think most of them if not all are 4bpp uncompressed:

Code: [Select]

4A1A40 4A20DF GFX Wallpaper 1 (1696 bytes) <- require wallpaper tile order
4A34DC 4A3B7B GFX Wallpaper 2 (1696 bytes)
4A3CDC 4A437B GFX Wallpaper 3 (1696 bytes)
4A44DC 4A4B7B GFX Wallpaper 4 (1696 bytes)
4A4CDC 4A537B GFX Wallpaper 5 (1696 bytes)
4A54DC 4A5B7B GFX Wallpaper 6 (1696 bytes)
4A5CDC 4A637B GFX Wallpaper 7 (1696 bytes)
4A64DC 4A6B7B GFX Wallpaper 8 (1696 bytes)
58FF64 5CF9A3 GFX     Monster / Esper Sprites (Variable * 459 entries)  <- require mold data for monsters / espers
5FE83E 60301D GFX Portraits (800 bytes * 23 entries) <- require tiles arrangement to display properly

GBA ROM maps: https://www.ff6hacking.com/wiki/doku.php?id=ff6a:ff6a
wallpaper tiles order: https://www.ff6hacking.com/wiki/doku.php?id=ff6a:data:formats:wallpaper
portrait tiles order: https://www.ff6hacking.com/wiki/doku.php?id=ff6a:data:formats:portrait

Edit: I had forgot the palettes. Here they are.. 

Code: [Select]

5FE55E 5FE83D PAL Portrait Palettes (16 colors * 23 entries)
6E7AD0 6EAAF8 PAL Monster Palettes (16 bytes * 764 entries)
6ED1B0 6ED2AF PAL Wallpaper Palettes (32 bytes * 8 entries)

I have some palettes of interest only documented in the (U) release:

Code: [Select]

6FD8E6 6FDCE6 PAL Map Sprite Palettes

GFX:
760000 792FFF SPRITE Character/Various Sprites (Variable * ??? entries)


[Squall]

This is a really cool project! Which games outside FF4, FF5 and FF6 do you plan to support?

Thank you! Actually this tool is not connected with any game. I just use FFs as base to test functionality :D

Thank you very much for the info, Madsiur! It was helpful to test a new Template - GBA 4bpp (no compression).
Unfortunately all the GFX require a tile-map, something that I have not implemented yet, so I couldn't see them properly.

I did notice that FF6a barely uses compression (at least not LZ10). Also all GFX uses a tile-map ... and tiles look scrambled inside ... Is there a particular reason for that, like weird copy protection of GFX? or just hmm bad programming :D

Anyway thanks to your data, version 2.0 is out! Download from first post.

P.S. Madsiur since FF6 uses LZSS variation do you know is it the same as FF5 LZSS one?

[Madsiur]

I did notice that FF6a barely uses compression (at least not LZ10).

It use compression for things that are not used often it seems  like ending cut-scenes, some things like the Magitek Cart ride. Compressed GFX are labeled on the wiki ROM maps.

Also all GFX uses a tile-map ... and tiles look scrambled inside ... Is there a particular reason for that, like weird copy protection of GFX? or just hmm bad programming :D

Well a tilemap can be a form of compression. For monsters, it removed the possible empty tiles in the mold (32x32 or 64x64). As for something like the Window backgrounds, it's better to have 31 window tiles not ordered than having repetitions in a multiples of well aligned patterns. Tile order for sprites is also a way to prevent tile repetitions in sprite poses. However for something like portraits, I have no idea while tiles were put this way... Maybe reading them in batch of 4 (?) resulted in this being the optimal way to dispose them?

Madsiur since FF6 uses LZSS variation do you know is it the same as FF5 LZSS one?

That I have no idea, but it worth a try!

[assassin]

i dunno how many types of compression are in use.  the only LZSS function i'm familiar with is C2/FF6D (but there are reportedly others):
http://assassin17.brinkster.net/code2i.txt
http://www.romhacking.net/utilities/1176/

something that came up in my search for the latter link was "Peer Sprite Viewer":
http://geigercount.net/crypt/index.html

[Squall]

Code: [Select]

Read next byte from the stream:

 If the bit is 1:
   Read next byte from $D0
   Write it on the circular buffer
   Write it on $D3
 If the bit is 0:
   Read 16 bit value from $D0
   That value contains an 'offset' and a 'length': O07 O06 O05 O04 O03 O02 O01 O00 | O10 O09 O08 L04 L03 L02 L01 L00
   Do (length + 3) times
       Read a byte from the circular buffer with the given offset
       Write the byte on the circular buffer
       Write the byte on $D3
       Increment offset (if offset > #$07FF then offset = #$0000)
After consuming the 8 bits from the bitmap byte, a new bitmap byte is read from $D0.
This is the FF5 LZSS algorithm. It uses $800 byte circular buffer. The repetition is 2 bytes - 11b Offset, 5b Length.

Thank you for your notes assassin! It seems to me its the same as your description. The only difference I see is that the Length (quantity as you call it) is placed in bits 3-7, while in FF5 its 0-4. Is there any chance you count bits from left to right? If so than its the same :D

[Squall]

Version 2.2 is out!
 - Added Template: SNES 3bpp (no compression)
 - Added Template: SNES 4bpp (no compression)
 - Major workflow change
 - Stability improvement

P.S. I need help to test 8bpp GBA graphics. If you know the addresses of graphic tiles of 8bpp in any GBA ROM (non-compressed or LZ77-10) please share some and the name of the ROM. If you happen to know their palette address that will be even better :thumbsup:
Do you know games for GBA that use direct-color (non palette)?
Do you know GBA games that internally store graphics in bpp different of 4/8? I wonder is it worth to implement other BPPs for GBA?

[Squall]

Short update:
 - I was able to implement LZSS version for FF6s.
 - Since now I have 3 methods of decompression, plenty ob BPP format, and number of ways tiles are laid in the memory, probably I will have to break improved Templates, so the user can choose separately BPP and Compression methods (otherwise it will be a long list to choose from). That was actually my initial idea, but that will require again major internal rework. That is the reason for no new release :(

On positive side: yes now I can decompress LZSS-FF6!!! assassin, thank you very much for your notes and provided links!!! They were invaluable in order to see the difference with FF5 and actually implement it.
For those that are interested:
 - both FF5&FF6 uses LZSS with sliding window of $800 bytes (external), using 11b Offset and 5 b Length
 - small difference - the bits for Offset:Length pair is coded in different place in 2B (16b)
 - BIG difference - FF5 uses decompressed length as marker to end, while FF6 uses compressed length

[Mauron]

Playing with the compression menu, when I changed to GBA 4BPP LZ10 on something not in that format, I got both a friendly "Not valid LZ10 stream", and a more technical "Access violation at address 0047F09E in module 'VisualSAK.exe'. Read of address 00000006."

[Squall]

Thank you Mauron for reporting this. Could you tell me more about the setting where  the bug appear? Probably a screenshot will be the best.

[Mauron]

Screenshots of both messages

I purposely chose an invalid template for that data. It looks like it's not stopping after the first error message.

[Squall]

Screenshots of both messages

I purposely chose an invalid template for that data. It looks like it's not stopping after the first error message.

Yeah no stopping ... I will work on that before releasing the new version. It seems I will have to separate compression from Template (the way it was originally designed) because the Template will grow exponentially with new bpp and compression methods added.

Thank you for finding that bug and reporting it, Mauron! Would you be so kind and share the impression from first time, no help/tutorial experience?

BTW you seem experienced in CT, do you know what compression method uses SNES version (it seems different from FF6)? I would love to implement it and add it to the tool :)

[Mauron]

Here's a document on Chrono Trigger's compression.

It seems like a good start. I would find it useful if I could load parts of a palette from different locations in a ROM, since Chrono Trigger handles weapon palettes that way, but I'm not sure how useful that would be overall.