Editing GimConv


=Description=
'''GimConv.exe''' is a sony program for microsoft windows used to convert GIM images, see [[Graphic Image Map (GIM)]]

*GimConv.exe versions:
**'''1.20c''' (GIM files made around 2005 included in pre-retail PS3 firmwares such 0.82.006)
**'''1.20e''' (GIM files made around 2006 and still present in PS3 4.82 firmware)
**'''1.20h''' released with '''p3tcompiler v2.00''' as part of the [http://uk.playstation.com/ps3/support/settings/detail/linked235336/item85346/Download-PlayStation-3-Custom-Theme-Guidelines/ PS3 Custom Theme Toolbox] (this is the version included in rcomage using a modifyed GimConv.cfg)

*Other notes:
**GIM images can be previewed in PC by using '''Gim Viewer''' (unnofficial app made by hsreina, it has limited support for the "rgba based" GIM formats). See [[Multimedia_Formats_and_Tools#GIM | Multimedia Formats and Tools]]
**The option '''-viewer''' (start gimview.exe when process ends) in the official config file (that was removed for rcomage) is intended to launch an official '''gimview.exe''' that doesnt exists in the public release
**[http://us.playstation.com/support/manuals/psp/pspguides/ PSP Custom Theme Toolbox] admits GIM files as inputs, but doesnt includes a GIM converter
**Some of the settings (such '''merge_mode''') are related with animated images but the support of animated GIM images on PlayStation consoles is unknown, see: http://endlessparadigm.com/forum/showthread.php?tid=5054
**It supports AVI video as input, probably to convert the video frames to static images and build a GIM animation with them, unknown usage
**Gimconv.exe supports an internal script format to automatize image conversions, it can be seen how appears as an option available at bottom of Gimconv.cfg and allows to create GIM-to-GIS conversions. The usage of this scripting features is mostly unknown
**There is a [http://tizzyt-archive.blogspot.com.es/2013/10/gis2png.html GIS2PNG converter] made by TizzyT

=GimConv.cfg Overview=
Before explaining how GimConv.exe works is needed an introduction about GimConv.cfg (the config file). All features of GimConv.exe are driven by the settings that appears on GimConv.cfg
 usage {
         This is the help screen (only informative)
 }
 default {
         Default predefined settings (loaded "from top to bottom", if a setting is repeated two or more times the most at bottom overrides the other/s at top)
 }
 input * {
         Settings for input file formats
 }
 option * {
         Groups of settings (overrides the defaults settings, are only used when you specify the option name at the command line)
 }
 script * {
         Unknown
 }

Below is shown GimConv.cfg released by sony as part of the [http://uk.playstation.com/ps3/support/settings/detail/linked235336/item85346/Download-PlayStation-3-Custom-Theme-Guidelines/ PS3 Custom Theme Toolbox] (official version)

The contents of GimConv.cfg are loaded "from top to bottom" first are loaded the default settings, and then the settings inside the option (incase you choose an option). The settings inside the options (at bottom ofGimConv.cfg) are actually "overriding" one or more default settings (at top of GimConv.cfg), this overriding also applyes to any setting or option you use in command line, as example, if you change the same setting 3 times in a single command line the only setting that counts is the last one
<div style="height:520px; overflow:auto">
{{Boxcode|content=<syntaxhighlight lang="c">
//================================================================
//  gimconv configuration
//================================================================

usage {
	puts "usage:"
	puts "        gimconv <input files> [options]"
	puts ""
	puts "options:"
	puts "        -interact       input additional options"
	puts "        -pictures       merge files as pictures"
	puts "        -frames         merge files as frames"
	puts "        -levels         merge files as levels"
	puts "        -prompt         prompt always"
	puts "        -warning        prompt on warning"
	puts "        -error          prompt on error"
	puts "        -viewer         start gimview.exe when process ends"
	puts "        -o <filename>   specify output file name"
	puts "        -s <w,h>        resize image data"
	puts "        -S              output text format"
	puts "        -P              resize image data to a power of two"
	puts "        -N              output in normal pixel storage format"
	puts "        -F              output in faster pixel storage format"
	puts "        -R              re-convert ( don't swap RGBA <-> ABGR )"
}

//----------------------------------------------------------------
//  default settings
//----------------------------------------------------------------

default {
	load "lib/*"

	//  main control

	interact_mode = off		// off on ctrl
	merge_mode = off		// off picture frame level plane
					// image palette sequence
	prompt_mode = off		// off on warning error
	output_object = on		// off on
	output_script = off		// off on
	output_directory = auto		// current input auto
	output_filename = ""
	object_extension = .gim
	script_extension = .gis
	userdata_extension = .dat
	glob_extension = .dds .tm2 .tga .bmp .avi
	object_viewer = ""

	//  process control

	filter_script = "Filter"
	image_size = default		// default pow2 $w,$h
	pixel_order = default		// default normal faster
	pixel_alpha = default		// default $alpha
	pixel_channel = default		// default rgba rgab ragb ... etc
	image_format = default		// default rgba8888 rgba4444 rgba5551 rgba5650
					// index4 index8 index16 index32
					// dxt1 dxt3 dxt5 dxt1ext dxt3ext dxt5ext
	palette_format = default	// default rgba8888 rgba4444 rgba5551 rgba5650
	update_userdata = default	// default off on
	update_fileinfo = off		// default off on

	check_limit = on		// off on
	limit_image_width = 4096	// off $count
	limit_image_height = 4096	// off $count
	limit_level_count = 8		// off $count
	limit_frame_count = off		// off $count

	//  output control

	format_style = psp		// std psp ps3
	format_endian = little		// little big

	output_image = on		// off on
	output_palette = on		// off on
	output_sequence = on		// off on

	//  PS3 OSD default settings

	format_endian = big		// -B
	pixel_order = normal		// -N
	pixel_channel = default		// -R
	image_format = rgba8888		// --image_format rgba8888

	extend_edge = off
}

//--------------------------------------------------------
//  format specific
//--------------------------------------------------------

input *.avi {
	pixel_alpha = 255
}

input *.bmp {
	pixel_alpha = 255
}

input *.dds {
}

input *.tga {
}

input *.tm2 {
}

input *.gim {
}

input *.gis {
}

//--------------------------------------------------------
//  mode option
//--------------------------------------------------------

option -interact {
	interact_mode = on
}

option -pictures {
	merge_mode = picture
}

option -frames {
	merge_mode = frame
}

option -levels {
	merge_mode = level
}

option -planes {
	merge_mode = plane
}

option -images {
	merge_mode = image
}

option -palettes {
	merge_mode = palette
}

option -sequences {
	merge_mode = sequence
}

option -prompt {
	prompt_mode = on
}

option -warning {
	prompt_mode = warning
}

option -error {
	prompt_mode = error
}

option -viewer {
	object_viewer = "GimView.exe"
}

//--------------------------------------------------------
//  process option
//--------------------------------------------------------

option -o %filename {
	output_filename = %filename
}

option -s %size {
	image_size = %size
}

option -g %ext {
	glob_extension = %ext
}

option -P {
	image_size = pow2
}

option -N {
	pixel_order = normal
}

option -F {
	pixel_order = faster
}

//--------------------------------------------------------
//  PS3 OSD specific options
//--------------------------------------------------------

option -B {
	format_endian = big
}

option -X {
	pixel_channel = abgr
}

option -R {
	pixel_channel = default
}

option -DXT {
	pixel_order = default
	pixel_channel = default
	image_format = default
}

option -E {
	extend_edge = on
}

//--------------------------------------------------------
//  output option
//--------------------------------------------------------

option -S {
	output_object = off
	output_script = on
}

//--------------------------------------------------------
//  filter script
//--------------------------------------------------------

script Filter %target {
	RescaleImage   %target $image_size
	SetPixelFormat %target $image_format $palette_format
	SetPixelAlpha  %target $pixel_alpha
	SwapPixelChannel %target $pixel_channel
	ConfigureImage %target order $pixel_order
	UpdateUserData %target $update_userdata
	UpdateFileInfo %target $update_fileinfo
	ExtendEdge     %target $extend_edge
	CheckPSP       %target $check_limit
}
</syntaxhighlight>}}
</div>

The config file supplyed by [[Rcomage]] contains some changes in the default settings that was common for PSP (included a size limit that breaks compatibility with PS3 and other problematic default settings), some custom options for the formats that are used in PSP, and a option was added for PS3 (labeled '''-ps3''') that seems to be wrong. For more info about the modifications made in the config file supplyed with [[rcomage]] see the differences in the {{talk}} page

=GimConv.exe usage=

==Help screen==
The help screen of GimConv.exe displays a few options with an small explain for them, the example below belongs to the official GimConv.cfg with the options sony considered useful, this options are mostly useless for us, in short: interact doesnt allows to do special things, the merge modes are hardcore because are intended to create GIM files by some kind of artisan method where you create the GIM by using "pieces" from other files, the promt modes doesnt helps much, gimview.exe doesnt exists, the image resize methods should not be used (instead you should prepare your images with the correct size before processing them with GimConv.exe), the pixel formats normal/faster should not be used that way individually, and the rgba color channels swapping are not needed for PS3

{{Keyboard|content='''C:\PS3_Custom_Theme_v200\GimConv>GimConv.exe'''
gimconv ver 1.20h --- generic picture converter
 Copyright (C) 2006 Sony Computer Entertainment Inc.
  All Rights Reserved.

usage:
        gimconv <input files> [options]

options:
        -interact       input additional options
        -pictures       merge files as pictures
        -frames         merge files as frames
        -levels         merge files as levels
        -prompt         prompt always
        -warning        prompt on warning
        -error          prompt on error
        -viewer         start gimview.exe when process ends
        -o <filename>   specify output file name
        -s <w,h>        resize image data
        -S              output text format
        -P              resize image data to a power of two
        -N              output in normal pixel storage format
        -F              output in faster pixel storage format
        -R              re-convert ( don't swap RGBA <-> ABGR )
}}

==Options==
{{Boxtip1|content=To specify an option is needed to use '''a single dash''' with the name of the option.}}
GimConv.exe allows to make a GIM-to-PNG conversions just by specifying the file extension with the option '''-o''':
{{Keyboard|content='''gimconv.exe in.gim -o out.png'''}}

If you create a PNG-to-GIM (or any of the supported input formats to GIM) without specifying any other option are going to be loaded all the default settings of GimConv.cfg (from top to bottom) and this is not good, most probably is going to generate a GIM file not compatible with PS3, you can decide to use a modifyed GimConv.cfg with all the default settings required by PS3 but this is not good enought either because there are (at least) 2 different GIM formats used in PS3, and you are going to break the compatbility with PSP, anyway here is the command as an example
{{Keyboard|content='''gimconv.exe in.png -o out.gim'''}}

At the time GimConv.exe processes the command it loads the settings of GimConv.cfg file "from top to bottom" (so it starts loading the default settings) and then it loads the option we have choosen (and this option contains settings that overrides the defaults)
{{Keyboard|content='''gimconv.exe in.png -o out.gim -DXT'''}}
The option '''-DXT''' used in this example contains 3 settings that are going to be loaded after the default settings (this is an example of the official GimConv.cfg, the one used inside rcomage is different):
{{Boxcode|content=<syntaxhighlight lang="c">
option -DXT {
	pixel_order = default
	pixel_channel = default
	image_format = default
}
</syntaxhighlight>}}

Another example where can be seen better how the options contains settings that overrides the default settings:
{{Keyboard|content='''gimconv.exe in.gim -o out.gis -S'''}}
*The option '''-S''' contains this settings:
{{Boxcode|content=<syntaxhighlight lang="c">
option -S {
	output_object = off
	output_script = on
}
</syntaxhighlight>}}
At top of GimConv.cfg (as part of the default settings) appears the settings: '''output_object''' with value '''on''' and '''output_script''' with value '''off''' (so by default gimconv.exe outputs a GIM and doesnt outputs a GIS). The option '''-S''' of this example is switching them (doesnt outputs a GIM but it outputs a GIS)

==Settings==
{{Boxtip1|content=To specify a setting is needed to use '''a double dash''' with the name of the setting.}}
This is the most accurate way to have full controll of how the GIM is built, if you are picky you can write a long line of settings to be sure you are overriding all the defaults (just incase the GimConv.cfg was modifyed by someone, by doing this you are overriding his/her changes made to default settings). If you are writing an script or any tool that sends actions to gimconv.exe you should use this method
{{Keyboard|content='''gimconv.exe in.png -o out.gim --pixel_order normal --format_style psp --format_endian big'''}}
The 3 settings used in the example are overriding the default settings at top of GimConv.cfg (this is an example of the official GimConv.cfg, the one used inside rcomage is different):
{{Boxcode|content=<syntaxhighlight lang="c">
//----------------------------------------------------------------
//  default settings
//----------------------------------------------------------------
	pixel_order = default		// default normal faster
	format_style = psp		// std psp ps3
	format_endian = little		// little big
</syntaxhighlight>}}

==Cumulative options and settings==
{{Boxtip1|content=Reminder: options are passed with '''a single dash''' and settings with a '''double dash'''}}
This is the official way, the predefined options of the official GimConv.cfg are intended to be used together with other options, this way you can have an option containing a group of settings (that are common for a console) and several other options for the different GIM formats supported in that console, using GimConv.exe this way is a bit more confusing though and makes harder to pass commands to it, but is very handy to make tests
{{Keyboard|content='''gimconv.exe in.png -o out.gim -DXT --pixel_channel brag'''}}
The command is loading the option '''-DXT''' like in the example above, but is overriding one of the settings inside '''-DXT''', with the '''--pixel_channel brag''' to reorder the color channels (from left to right, BLUE, RED, ALPHA, GREEN), the resulting GIM is going to be like the one made in the example above, the only difference is the location of the pixel colors

It allows to cummulate as many settings and options you want, in any order (be careful this could create some problems but it seems it works fine) to do something like this:
{{Keyboard|content='''gimconv.exe in.png -o out.gis -DXT --pixel_channel brag -S'''}}

==From rcomage==
{{Boxwarning1|content='''Dont use this method for PS3'''. GIM files inside RCO files from PS3 firmware uses variated formats, this method applyes the same settings for all them and breaks some}}
This works by sending GIM settings from rcomage.exe to gimconv.exe and is made at the time rcomage.exe is compiling the RCO (see the "compile" help in [[rcomage]] page)

There are 2 options in rcomage specific for this task, named: '''--gimconv-cmd''' and '''--gimconv-flags'''
{{Keyboard|content='''rcomage.exe compile in.xml out.rco --gimconv-cmd ".\gimconv\gimconv.exe" --gimconv-flags "--pixel_order normal --format_style psp --format_endian big"'''}}

Inside RCO files from PS3 official firmwares there are (at least) 2 types of GIM formats, and both can be used inside the same RCO, as example... an RCO with a total of 14 GIM files and 12 of made using some GIM settings, and the other 2 with different GIM settings, for this reason is needed to handle every GIM file individually

Is not posible to compile a PS3 RCO with rcomage by using PNG images as source because rcomage.exe sends the GIM settings to gimconv.exe to make the PNG--->GIM image conversions and this GIM settings are applyed to all PNG images, so all the GIM files in the RCO will have the same GIM settings, this is wrong

<!--
In other words, the only way to compile this a RCO where you want to modify some GIM files is in 2 steps. In the first step is needed to make the different GIM conversions with gimconv.exe (and all the other conversions in case exists .VAG audio a WAV2VAG conversion or a JS2VSMX for the .VSMX script). In the second step is needed to compile the RCO with rcomage.exe by using the GIM's VAG's and VSMX's as sources without any kind of file conversions
-->

=GimConv.cfg options for PS3=
You can modify the GimConv.cfg file by adding some options (with custom names) containing any settings, by doing this you can keep a record of the correct settings for some GIM formats and group lot of settings under a single option, additionally when using GimConv.exe from command line the commands will be much shorter. This wiki section proposes a couple of options (but probably PS3 supports a lot more) that have been tested and are the first and second most commonlly used in PS3 official firmwares

*'''1st most common''' GIM format used in official PS3 firmwares
{{Boxcode|content=<syntaxhighlight lang="c">
option -ps3rgba8888 {
	format_endian = big
	format_style = psp
	image_format = rgba8888
	pixel_order = normal
	pixel_channel = rgba
	limit_image_width = 4096
	limit_image_height = 4096
}
</syntaxhighlight>}}

*'''2nd most common''' GIM format used in official PS3 firmwares
{{Boxcode|content=<syntaxhighlight lang="c">
option -ps3dxt5 {
	format_endian = big
	format_style = psp
	image_format = dxt5
	pixel_order = normal
	pixel_channel = default
	limit_image_width = 4096
	limit_image_height = 4096
}
</syntaxhighlight>}}

=How to identify GIM settings=
The goal is to identify the GimConv settings that was used when the original GIM file was created, there are basically two ways to identify GIM settings, by reading the GIM structure, or by bruteforce

Both identification methods can be made manually or by using external code, and are not exclusive (for better accuracy you can do both)

==By reading GIM structure==
Long story short... the GIM format is not completly documented, it allows for lot of features that generates different GIM structures, being realistic... at this point is not posible to retrieve all info needed for rebuild purposes from the original GIM, for more info see: [[Graphic Image Map (GIM)]]
*Anyway, the list of settings needd to collect from the original GIM file are:
**'''format_endian''' (offset 0x0 in the GIM)
**'''format_style'''  (offset 0x8 in the GIM)
**'''image_format''' (offset 0x4 of block 0x4 in the GIM)
**'''pixel_order''' (offset 0x8 of block 0x4 in the GIM)

*Some notes:
**GIM files using image formats with indexed colors have an additional block 0x5, and block 0x4 is displaced
**The list of settings mentioned above will work '''only''' if the GIM contains a single image/level/frame, otherway the GIM structure becomes more complex
**There are 4 unknown fields in GIM structure, and it seems GIM was used in other sony devices that are unknown, so dont take this as something general, is just an approach

For this reasons, it cant be said this method is going to be 100% accurate, there is room for errors, the best way to solve this problem is by using the next method explained... by bruteforce

==By bruteforce==
{{Boxinfo1|content='''GIM-to-GIM''' is the only conversion method that is '''[https://en.wikipedia.org/wiki/Lossless_compression lossless]''' for all image formats supported by GimConv.exe}}
The word [https://en.wikipedia.org/wiki/Brute-force_search bruteforce] sounds a bit ridiculous used in this enviroment, but it defines pretty well how this identification method works. Consist in rebuilding the GIM file several times by using different GimConv settings until we generate a GIM file identical to the original. In other words... if you find how to create a GIM file exactly like the original you can be 100% sure the settings you are using are exactly the same used by sony to create that GIM

The only way (that works with all image formats) to create a GIM file exactly like the original is by doing a GIM-to-GIM conversion, this is not so retarded as it sounds, GimConv really rebuilds the file when doing a GIM-to-GIM conversion and in the process is preserved the pixel data of DXT based image formats, there are other rebuild method that allows to create GIM files identical to the original such GMI-to-PNG-to-GIM but only works for rgba based image formats, the point is GIM-to-GIM is the only that preserves pixel info for DXT formats, adnd for this reason the GIM-to-GIM conversion superceedes all other methods

In bruteforce is used a "dictionary" that contains the posible candidates that solves the "problem"... the good thing is our dictionary (with the GIM formats found in PS3) only contains 3 candidates, this means in the worst scenario you are going to need to rebuild the GIM file 3 times... if none of the 3 GIM files matches with the original we have a problem (please report back and we will add the rare format to this wiki page)


*GIM--->GIM method (prefered)
**Use gimconv with .gim files as input and output (this preserves dxt pixel info, because dxt images doesnt suffers a conversion)
'''1st most common'''
{{Keyboard|content='''gimconv.exe original.gim -o try1.gim --ps3rgba8888'''}}
'''2nd most common''' (this is a variant of the previous)
{{Keyboard|content='''gimconv.exe original.gim -o try2.gim --ps3rgba8888 --update_fileinfo on'''}}
'''3rd most common'''
{{Keyboard|content='''gimconv.exe original.gim -o try3.gim --dxt5'''}}

*GIM--->PNG--->GIM method (fails when there is involved a DXT conversion because DXT conversions are lossy, works fine for rgba**** image formats):
**Use rcomagegui to make an extraction of the contents '''without''' conversion of images (this way the GIM are extracted as GIM)
**Use rcomagegui to make another extraction of the contents '''with''' conversion of images (this way the GIM are extracted as PNG)
**Use gimconv to convert a PNG ---to---> GIM by the old method of test-error by trying some of the options availables that appears in gimonv.cfg
**Use a hexeditor to compare the original GIM extracted in step 1 with the rebuilt GIM from your test, there is no need to make accurate comparisons, just an overview of the first 0x80 bytes of the header, and how the pixel data is ordered starting at offset 0x80
**When the header and pixel data seems similar make a hash comparison, original GIM and rebuild GIM should be exactly the same hash (is posible to create a GIM exactly like the original)

<!-- GimConv.exe version patch
This allows to create GIM files identical to some of the ones included in PS3 firmware by doing a GIM-to-GIM conversion
The goal of this is to identify accuratelly the original GIM settings with some kind of bruteforce method (the classic test and error, but only with 3 posible candidates, so is relativelly easy and works pretty well)

The patch applyed to GimConv.exe changes the version... the reason to do this is because this version is going to be included in the rebuilt GIM, at the bottom as part of the "fileinfo" (block id 0x00FF)

GimConv.exe v1.20h original (MD5:425B2E4D9684BCE1FC79A37247618AC8)
At absolute offset 0x91E4 replace 0x68 by 0x65 (character 'h' by character 'e')
GimConv.exe v1.20h patched (MD5:6AFA9F10221CDDACE3E294D13A572250)

With the patched GimConv.exe you can run this command:
GimConv.exe original.gim -o rebuilt.gim -ps3bpp32 --update_fileinfo on
This preserves the original "fileinfo", so the rebuilt.gim will have identical fileinfo than the original.gim

As far i know all the GIM files using "fileinfo" from PS3 retail firmwares was build with GimConv.exe v1.20e so it should work... if at some point someone finds other GIM files that was made with with other GimConv.exe versions (f,g,h) and you want to use this identification process you can make another GimConv.exe patched with different versions and rename them to GimConvE.exe, GimConvF.exe, GimConvG.exe etc... and have all them in the same folder next to the original v1.20h

Instead of applying this patch this can be automatized with scripts or external code, but the concept of how this identification method works is the same, the goal is to create a GIM exactly like the original by doing a GIM-to-GIM conversion and compare hashes of them... if original GIM and rebuilt GIM are identical you found the settings

-->

==Exercises (wip:)==
*In [[explore_plugin]]_full.rco from firmware 4.70 there are 245 images:
**201 was created by using settings: ''--format_style psp --format_endian big --image_format rgba8888 --pixel_order normal'' (exact match of gims after rebuilding gim--->png--->gim), such tex_psn.gim
**030 was created by using settings: ''--format_style psp --format_endian big --image_format rgba8888 --pixel_order normal --update_fileinfo on'' (exact match of gims after removing footer from originals and rebuilding gim--->png--->gim), see: [[Graphic_Image_Map_(GIM)#0xFF_.28Fileinfo.29|fileinfo]]:
**014 was created by using settings: ''--format_style psp --format_endian big --image_format dxt5'' (exact match of gims after rebuilding gim--->gim)
***item_tex_trophy.gim, trophy_tex_grade_bronze.gim, trophy_tex_grade_gold.gim, trophy_tex_grade_hidden.gim, trophy_tex_grade_platinum.gim, trophy_tex_grade_silver.gim, item_tex_CardBallon.gim, item_tex_Profile_Game.gim, item_tex_Profile_Highlight.gim, item_tex_Profile_Onlines.gim, tex_indi_Me.gim, tex_sb_base.gim, tex_sb_base_s.gim, tex_sb_slider.gim

<!--
{| class="wikitable"
|-
! image group or rco !! gim settings !! notes
|-
| '''ps3rgba8888''' || --format_style psp --format_endian big --image_format rgba8888 --pixel_order normal || Most used GIM format inside .rco's
|-
| '''ps3dxt5''' || --format_style psp --format_endian big --image_format dxt5 || Second most used GIM format inside rco's
|}
-->

{{File Formats}}
<noinclude>[[Category:Main]]</noinclude>