Editing CXML Containers

Jump to navigation Jump to search
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.

Latest revision Your text
Line 45: Line 45:
};
};


struct ChildElementBin
struct ElementBin
{
{
int name;
int name;
Line 55: Line 55:
int last_child;
int last_child;
};
};
       
 
        struct BrotherElementBin
{
int name;
int attr_num;
int parent;
int prev;
int next;
};
       
struct AttributeBin
struct AttributeBin
{
{
                int INTEGER = 1;
                int FLOAT = 2;
                int STRING = 3;
                int INTEGER_ARRAY = 4;
                int FLOAT_ARRAY = 5;
                int DATA = 6;
                int ID_REF = 7;
                int ID_NO_REF = 8;
int name;
int name;
int type;
int type;
Line 186: Line 169:


===Element===
===Element===
{| class="wikitable" style="text-align:center;"
|+ CXML Child Element definition
! Offset !! Length !! usage
|-
| 0x00 || 0x04 || Element name relative offset in '''TAGS Table'''
|-
| 0x04 || 0x04 || Number of attributes in '''TOC'''
|-
| 0x08 || 0x04 || Parent relative offset in '''TOC'''
|-
| 0x0C || 0x04 || Previous Brother relative offset in '''TOC'''
|-
| 0x10 || 0x04 || Next Brother relative offset in '''TOC'''
|-
| 0x14 || 0x04 || First Child relative offset in '''TOC'''
|-
| 0x18 || 0x04 || Last Child relative offset in '''TOC'''
|}


{| class="wikitable" style="text-align:center;"
===Attribute===
|+ CXML Brother Element definition
 
! Offset !! Length !! usage
{| class="wikitable"
|-
|+ Attribute types structure
| 0x00 || 0x04 || Element name relative offset in '''TAGS Table'''
! rowspan=2 | Offset !! rowspan=2 | Length !! rowspan=2 | Name !! colspan=8 | Attribute type
|-
| 0x04 || 0x04 || Number of attributes in '''TOC'''
|-
| 0x08 || 0x04 || Parent relative offset in '''TOC'''
|-
| 0x1C || 0x04 || Previous Brother relative offset in '''TOC'''
|-
|-
| 0x10 || 0x04 || Next Brother relative offset in '''TOC'''
! Integer !! Float !! String !! Integer Array !! Float Array !! File !! ID with ref !! ID without ref
|}
*NOTE: The '''Child Element''' defination describe the Child Element and his Attributes where the '''Brother Element''' describe the Brother Element and his Attributes.
So the full order to read will be '''Child Element''' >>> '''Attributes''', '''Brother Element''' >>> '''Attributes'''. The Root (so offset 0x00) of any SonyPlaystation Resource file always start's with a '''Child Element''' to read.
 
===Attribute===
{| class="wikitable" style="text-align:center;"
|+ CXML Attribute definitions
! Attribute name<BR />(offset 0x0, length 0x4)
| colspan=8 | Attribute name relative offset in '''TAGS Table'''
|-
|-
! Attribute type<BR />(offset 0x4, length 0x4)
| 0x0 || 0x4 || Attribute name offset || colspan=8 {{cellcolors|lightgrey}}  '''________________________________________________ xml tag string from the "strings table" ____________________________________________'''
! 1<BR />(Single Integer) !! 2<BR />(Single Float) !! 3<BR />(String) !! 4<BR />(Integer Array) !! 5<BR />(Float Array) !! 6<BR />(File) !! 7<BR />(ID with loopback ref) !! 8<BR />(ID with crossed loopback ref)
|-___
| 0x4 || 0x4 || Attribute type || 1 || 2 || 3 || 4 || 5 || 6 || 7 || 8
|-
|-
! variable 1<BR />(offset 0x8, length 0x4)
| 0x8 || 0x4 || variable 1 || Integer value || Float value || String Offset || Integer Array Offset || Float Array Offset || File offset || Filename offset || Filename offset
| Integer value || Float value || '''TAGS Table''' offset || '''Integer Arrays Table''' offset || '''Float Arrays Table''' offset || '''Files Table''' offset || '''Strings Table''' offset || '''Strings Table''' offset
|-
|-
! variable 2<BR />(offset 0xC, length 0x4)
| 0xC || 0x4 || variable 2 || unknown (usually 0) || unknown (usually 0) || String Length || Integer count || Float count || File size || unknown (usually 0) || unknown (usually 0)
| {{cellcolors|#ffaaaa}} Not used <abbr title="(always 0 in the samples found)">(always 0)</abbr> || {{cellcolors|#ffaaaa}} Not used <abbr title="(always 0 in the samples found)">(always 0)</abbr> || String length || Number of integer's to read || Number of float's to read || File size || {{cellcolors|#ffaaaa}} Not used <abbr title="(always 0 in the samples found)">(always 0)</abbr> || {{cellcolors|#ffaaaa}} Not used <abbr title="(always 0 in the samples found)">(always 0)</abbr>
|}
|}
*Attribute types 1 and 2 are the only ones that stores the value inside the attribute definition itself (from the point of view of code are defined and given a value at the same time, from the point of view of the structure can be said the value is "embedded" in the cxml TOC). Attributes 1 and 2 doesnt accesses any of the other tables of the cxml structure
*Attribute types 4 and 5 stores an array of values inside an "arrays table". This tables stores multiples arrays in a special format intended to reduce the final size of the table by avoiding duplicated arrays inside it. To recover a single array from a "arrays table" attributes types 4 and 5 specifyes a position in the table by the "offset" (where the array we are trying to recover begins) and the "number" (of consecutive items that composes the array we are trying to recover) <!-- this arrays table access needs a better explain anyway, because is important to know how works the build process that removes duplicated values -->
*Attribute type 7 stores a string in the "strings table", in this table next to the string we are trying to recover is defined an offset that "loops back" to the element that owns the attribute.
**Example: in [http://www.psdevwiki.com/ps3/CXML_Containers#QRC_2 rhm.qrc v1.00] the first "file" element of the TOC (at offset 0x78) has 2 attributes: "src" (at offset 0x94) is a type 6 attribute, and "id" (at offset 0xA4) is a type 7 attribute, "src" points to the "file table" and defines the file by his offset and size (this way you know where is the file, no mistery with attribute type 6). And "id" points to the string table at offset 0x1B0. If you look in the string table, the first string has an offset that points back to the "file" element in the TOC at offset 0x78 (in the runtime we have readed the definition of the element "file" in the TOC----> then the attribute id with the string---> then back to the TOC to the same "file" element so we can read all his attributes again).
**The purpose of this pointer that returns back from the strings table ---> to the TOC is a bit unknown, seems to be related with the other attributes owned by the same element and the way how the structure is accessed in the runtime (maybe the system does a loop to access every one of the attributes that belongs to an element). This special feature is what has been named in the wiki page before as "ID with ref" (and "ID without ref" for att type 8 <strike>because doesnt have it</strike>), now we are using too the concept of loops, loopback, and crossed loopbacks, but in my oppinion the names of attibute types 7 and 8 cant be decided yet because we dont know exactly what is the purpose of it (is hard to give it a name before knowing exactly how it works)


==Container Common Structure==
==Container Common Structure==
Line 270: Line 213:


*Notes
*Notes
**The first 2 bytes of the compressed QRCF is the ZLIB header, being the default/max level 9.
**The first 2 bytes of the compressed QRCF is the ZLIB header, in this example: 0x78DA (b: 0111 1000  1101 1010 / 30938) is using the default/max level 9.
 
**source: http://tools.ietf.org/html/rfc1950#section-2.2
<small>
<small>
{{Zlib Header}}
{| class="wikitable" style="text-align: center;"
|-
!width="100"|byte index
!width="60"|bit 7
!width="60"|bit 6
!width="60"|bit 5
!width="60"|bit 4
!width="60"|bit 3
!width="60"|bit 2
!width="60"|bit 1
!width="60"|bit 0
|-
|[0] '''CMF'''
|colspan="4"|'''Compression info'''
*7 = indicates a 32K window size
|colspan="4"|'''Compression method'''
*8 = denotes the "deflate" compression
|-
|[1] '''FLG'''
|colspan="2"|'''FLEVEL'''
* 0 - compressor used fastest algorithm
* 1 - compressor used fast algorithm
* 2 - compressor used default algorithm
* 3 - compressor used maximum compression, slowest algorithm
|colspan="1"|'''FDICT'''
(Preset dictionary)
If set, a DICT dictionary identifier is present
immediately after the FLG byte. The dictionary is a sequence of
bytes which are initially fed to the compressor without
producing any compressed output. DICT is the Adler-32 checksum
of this sequence of bytes (see the definition of ADLER32
below).  The decompressor can use this identifier to determine
which dictionary has been used by the compressor.
|colspan="5"|'''FCHECK'''
value must be such that CMF and FLG, when viewed as
a 16-bit unsigned integer stored in MSB order (CMF *256 + FLG),
is a multiple of 31.
|-
|}
</small>
</small>


Line 301: Line 282:
| Magic || 0x00 || 0x04 || {{cellcolors|#666666|#ffffff}} 51 52 43 46 || 'QRCF'
| Magic || 0x00 || 0x04 || {{cellcolors|#666666|#ffffff}} 51 52 43 46 || 'QRCF'
|-
|-
| Version || 0x04 || 0x04 || {{cellcolors|#888888|#ffffff}} 00 00 01 10 || CXML version '1.10'
| Version || 0x04 || 0x04 || {{cellcolors|#888888|#ffffff}} 00 00 01 10 ||
|-
|-
| Tree Table Offset || 0x08 || 0x04 || {{cellcolors|#dddd66}} 00 00 00 40 ||  
| Tree Table Offset || 0x08 || 0x04 || {{cellcolors|#dddd66}} 00 00 00 40 ||  
Line 370: Line 351:
| 0x30 || 0x04 || 00 00 00 38 || First Entry
| 0x30 || 0x04 || 00 00 00 38 || First Entry
|-
|-
| 0x34 || 0x04 || 00 00 01 28 || Last Entry
| 0x34 || 0x04 || xx xx xx xx || Last Entry
|-
|-
| 0x38 || 0x04 || 00 00 00 0F || String Pointer (Offset within '''String Table''', in this case it will be a Element <file>)  
| 0x38 || 0x04 || 00 00 00 0F || String Pointer (Offset within '''String Table''', in this case it will be a Element <file>)  
Line 400: Line 381:
| 0x6C || 0x04 || 00 00 00 00 || Offset of filename within the ID Table
| 0x6C || 0x04 || 00 00 00 00 || Offset of filename within the ID Table
|-
|-
| 0x70 || 0x04 || 00 00 00 00 || not used
| 0x70 || 0x04 || 00 00 00 00 || on icontex here are additional 16 bytes
|-
| 0x74 || 0x04 || 00 00 00 0F || String Pointer
|-
| 0x78 || 0x04 || 00 00 00 02 || Attribute Counter
|-
| 0x7C || 0x04 || 00 00 00 1C || Parent Element
|-
| 0x80 || 0x04 || 00 00 00 38 || Previous Element
|-
| 0x84 || 0x04 || 00 00 00 B0 || Next Element
|-
|-
|}
|}


On icontex.qrc there are additional 16 bytes added on offset 0x70 within the '''Tree Table''' because it has an additional attribute '''size'''
On icontex.qrc there are additional 16 bytes added on offset 0x70 within the '''Tree Table'''
{| class="wikitable"
{| class="wikitable"
! Offset !! Size !! Example !! Remark
! Offset !! Size !! Example !! Remark
|-
| 0x70 || 0x04 || 00 00 00 00 || on icontex.qrc
|-
|-
| 0x74 || 0x04 || 00 00 00 1B || String Pointer (Offset within '''String Table''', in this case it will be a Attribute <size>)
| 0x74 || 0x04 || 00 00 00 1B || String Pointer (Offset within '''String Table''', in this case it will be a Attribute <size>)
Line 414: Line 407:
| 0x7C || 0x04 || 00 01 55 D4 || on icontex.qrc (size of .dds uncompressed [Dec = 87508])
| 0x7C || 0x04 || 00 01 55 D4 || on icontex.qrc (size of .dds uncompressed [Dec = 87508])
|-
|-
| 0x80 || 0x04 || 00 00 00 00 || not used
| 0x80 || 0x04 || 00 00 00 00 || similar all qrc files
|-
| 0x84 || 0x04 || 00 00 00 0F || String Pointer
|-
| 0x88 || 0x04 || 00 00 00 02 || Attribute Count
|-
| 0x8C || 0x04 || 00 00 00 1C || Parent Element
|-
| 0x90 || 0x04 || 00 00 00 38 || Previous Element
|-
| 0x94 || 0x04 || 00 00 00 D0 || Next Element
|-
|-
|}
|}
Line 425: Line 428:
*For QRCF_2 and P3T themes only is stored the "id" (the file "src" is missing so there is no way to read the original path or file extension from the container)
*For QRCF_2 and P3T themes only is stored the "id" (the file "src" is missing so there is no way to read the original path or file extension from the container)


{{Boxcode|code=<syntaxhighlight lang="xml">
{{Boxcode|content=<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<qrc>
<qrc>
Line 1,118: Line 1,121:


*Example of the string table from frame_01.cxml
*Example of the string table from frame_01.cxml
{{Boxcode|code=<syntaxhighlight lang="xml">
root.version.1.00.jacket.id.normal.base.position.rotation.scale.name.frame_01.filetable.num.file.src.frame_01_lo.gim.lod.frame_01_parallax_512.gtf.shadow_square_1.gtf.shadow_square_2.gtf.shadow_3.gtf.frame_01_mi.gim.frame_01_hi.gim.vptable.vp.vp_jacket_ezprim.vpo.optional.vp_jacket_normalmap.vpo.vp_jacket_ezshadow.vpo.fptable.fp.fp_jacket_ezprim_rgb.fpo.fp_jacket_normalmap_specL.fpo.fp_jacket_ezshadow.fpo.modeltable.model.color.uv.vertex.tangent.binormal.layertable.layer.priority.type.image-frame-static.order.duration.image-thumbnail-auto
root.version.1.00.jacket.id.normal.base.position.rotation.scale.name.frame_01.filetable.num.file.src.frame_01_lo.gim.lod.frame_01_parallax_512.gtf.shadow_square_1.gtf.shadow_square_2.gtf.shadow_3.gtf.frame_01_mi.gim.frame_01_hi.gim.vptable.vp.vp_jacket_ezprim.vpo.optional.vp_jacket_normalmap.vpo.vp_jacket_ezshadow.vpo.fptable.fp.fp_jacket_ezprim_rgb.fpo.fp_jacket_normalmap_specL.fpo.fp_jacket_ezshadow.fpo.modeltable.model.color.uv.vertex.tangent.binormal.layertable.layer.priority.type.image-frame-static.order.duration.image-thumbnail-auto
</syntaxhighlight>}}


*Notes:
*Notes:
Line 1,140: Line 1,140:
In other words... the only difference between QRCF_1 and QRCF_2 is the added attribute '''size''' in every file entry that makes entries 16 bytes longer in QRCF_2... there is no need for an example of QRCF_2 because are almost the same
In other words... the only difference between QRCF_1 and QRCF_2 is the added attribute '''size''' in every file entry that makes entries 16 bytes longer in QRCF_2... there is no need for an example of QRCF_2 because are almost the same


This .xml represents structure of rhm.qrc 1.00, can be used to build it, and can be generated by reading the "tree table"


 
{{Boxcode|content=<syntaxhighlight lang="xml">
The .xml below represents the structure of [[rhm.qrc]] used in firmwares from 1.00 up to 1.32 ready to be compiled (compiled hash should be MD5: D1E7281246EC8A46EB6D368BD4B332BD)
{{Boxcode|code=<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<qrc>
<qrc>
<file-table>
<file-table>
<file src="C:/workdir/Clear_old.fpo" id="lib/rhm/Clear.fpo"/>
<file src="lib/rhm/Clear.fpo" id="lib/rhm/Clear.fpo"/>
<file src="C:/workdir/Clear_test12.vpo" id="lib/rhm/Clear.vpo"/>
<file src="lib/rhm/Clear.vpo" id="lib/rhm/Clear.vpo"/>
<file src="X:/DontCopyMe.fap" id="lib/rhm/Copy.fpo"/>
<file src="lib/rhm/Copy.fpo" id="lib/rhm/Copy.fpo"/>
<file src="D:/anotherworkdir/default_v12.fpo" id="lib/rhm/default.fpo"/>
<file src="lib/rhm/default.fpo" id="lib/rhm/default.fpo"/>
<file src="D:/anotherworkdir/default_v34.vpo" id="lib/rhm/default.vpo"/>
<file src="lib/rhm/default.vpo" id="lib/rhm/default.vpo"/>
</file-table>
</file-table>
</qrc>
</qrc>
</syntaxhighlight>}}
</syntaxhighlight>}}


When the file is compiled the info from the .xml is stored in the '''TOC''' (at offset 0x40, colored in red in the example below). It starts with the main element '''qrc''' and his only children '''file-table''' that has his own 5 childrens '''file''' (all them are elements). Every '''file''' element has 2 attributes '''src''' and '''id'''.
When the qrc container is build all the info from he .xml is stored in the "tree table" (at offset 0x40) that is the first area after the CXML header, and the files itself are stored in a "file table" placed at bottom of the container (at offset 0x250)


The attribute '''src''' stores the information needed to locate the file, either in PC (before compiled) or locally inside the CXML container (after compiled). In the .xml example above the value of '''src''' are PC paths (used by the compiler to locate the file inside the PC), but when compiled the value is lost, the file is stored inside the container '''Files table''' (at offset 0x250, colored in orange in the example below), and is indexed in the container '''TOC''' by using an entry with data type 6 composed of '''file offset''' + '''file size''' (used by PS3 firmware to locate the file inside the container '''Files table''')
The "tree table" (colored in red tones in the example) starts with the elements: "qrc" and "file-table"... and after this, the element "file" is repeated 5 times (first file entry at 0x78)... all this 5 "file" belongs to "file-table" element
 
The value of attribute '''id''' is stored inside the container '''Strings table''' (at offset 0x0x1B0, colored in green in the example below), and is indexed in the container '''TOC''' by using an entry with data type 7 composed by '''filename offset''' + '''unknown''' (used by the PS3 to locate the text string inside the container '''Strings table''')


{| class="wikitable" style="font-size:x-small;"
{| class="wikitable" style="font-size:x-small;"
Line 1,168: Line 1,165:
! rowspan=17 | CXML header
! rowspan=17 | CXML header
|- {{cellcolors|#666666|#ffffff}}
|- {{cellcolors|#666666|#ffffff}}
| 0x00 || 0x04 || QRCF || Magic ||  
| 0x00 || 0x04 || QRCF || Magic || unnofficially: QRCF_1
|- {{cellcolors|#666666|#ffffff}}
|- {{cellcolors|#666666|#ffffff}}
| 0x04 || 0x04 || 00000110 || Version ||  
| 0x04 || 0x04 || 00000110 || Version ||  
|- bgcolor="#ff8888"
|- bgcolor="#ff8888"
| 0x08 || 0x04 || 00000040 || TOC absolute offset, also header size || rowspan="2" | Stores CXML '''ELEMENTS''' and '''ATTRIBUTES''' definitions<br/ >and the '''VALUES''' of attribute types 1 ('''single integer'''), and 2 ('''single float''')
| 0x08 || 0x04 || 00000040 || Tree table absolute offset ||  
|- bgcolor="#ff8888"
|- bgcolor="#ff8888"
| 0x0C || 0x04 || 00000164 || TOC size  
| 0x0C || 0x04 || 00000164 || Tree table size ||
|- bgcolor="#88ff88"
|- bgcolor="#88ff88"
| 0x10 || 0x04 || 000001B0 || Strings table absolute offset || rowspan="2" | Stores the '''VALUES''' of XML attributes type 7 and 8
| 0x10 || 0x04 || 000001B0 || Filename table absolute offset ||  
|- bgcolor="#88ff88"
|- bgcolor="#88ff88"
| 0x14 || 0x04 || 00000071 || Strings table size  
| 0x14 || 0x04 || 00000071 || Filename table size ||
|- bgcolor="#8888ff"
|- bgcolor="#8888ff"
| 0x18 || 0x04 || 00000230 || TAGS table absolute offset || rowspan="2" | Stores the '''NAMES''' of XML [http://www.w3schools.com/xml/xml_syntax.asp '''elements''' and '''attributes''']]<br/ >and the '''VALUES''' of attribute types 3 ('''string''')
| 0x18 || 0x04 || 00000230 || Strings table absolute offset ||  
|- bgcolor="#8888ff"
|- bgcolor="#8888ff"
| 0x1C || 0x04 || 0000001B || TAGS table size  
| 0x1C || 0x04 || 0000001B || Strings table size ||
|- bgcolor="#ffeedd"
| 0x20 || 0x04 || 00000250 || Integer array absolute offset || ''Overlapped''
|- bgcolor="#ffeedd"
| 0x24 || 0x04 || 00000000 || Integer array size || ''Empty''
|- bgcolor="#ffeebb"
|- bgcolor="#ffeebb"
| 0x20 || 0x04 || 00000250 || Integer arrays table absolute offset || rowspan="2" | Stores the '''VALUES''' of XML attributes type 4 ('''integer array''')
| 0x28 || 0x04 || 00000250 || Float array absolute offset || ''Overlapped''
|- bgcolor="#ffeebb"
|- bgcolor="#ffeebb"
| 0x24 || 0x04 || 00000000 || Integer arrays table size  
| 0x2C || 0x04 || 00000000 || Float array size || ''Empty''
|- bgcolor="#ffee66"
|- bgcolor="#ffee99"
| 0x28 || 0x04 || 00000250 || Float arrays table absolute offset || rowspan="2" | Stores the '''VALUES''' of XML attributes type 5 ('''float array''')
| 0x30 || 0x04 || 00000250 || File table absolute offset || ''Overlaps the previous 2 tables''
|- bgcolor="#ffee66"
|- bgcolor="#ffee99"
| 0x2C || 0x04 || 00000000 || Float arrays table size
| 0x34 || 0x04 || 00000DD0 || File table size || ''Overlaps the previous 2 tables''
|- bgcolor="#ffaa00"
|- bgcolor="#ffee77"
| 0x30 || 0x04 || 00000250 || Files table absolute offset || rowspan="2" | Stores the '''FILES''' indexed in the TOC by attribute type 6 ('''file''')
| 0x38 || 0x04 || 00000000 || Reserved area absolute offset || ''Not used''
|- bgcolor="#ffaa00"
|- bgcolor="#ffee77"
| 0x34 || 0x04 || 00000DD0 || Files table size  
| 0x3C || 0x04 || 00000000 || Reserved area size || ''Empty''
|- bgcolor="#888888"
| 0x38 || 0x04 || 00000000 || Unknown area absolute offset || rowspan="2" | Unknown
|- bgcolor="#888888"
| 0x3C || 0x04 || 00000000 || Unknown area size  
|-
|-
! rowspan=98 | TOC
! rowspan=95 | Tree Table
|- bgcolor="#cccccc"
| colspan="5" |
|- {{cellcolors|#ff7777}}
|- {{cellcolors|#ff7777}}
| 0x40 || 0x04 || 00000000 || element name relative offset || '''qrc'''
| 0x40 || 0x04 || 00000000 || element name relative offset || '''qrc'''
Line 1,217: Line 1,212:
|- {{cellcolors|#ff7777}}
|- {{cellcolors|#ff7777}}
| 0x58 || 0x04 || 0000001C || last child element relative offset || 0x40 + 0x1C = '''last child at 0x5C absolute'''
| 0x58 || 0x04 || 0000001C || last child element relative offset || 0x40 + 0x1C = '''last child at 0x5C absolute'''
|- bgcolor="#cccccc"
|- {{cellcolors|#ff8888}}
| colspan="5" |
|- {{cellcolors|#ff8888}}
| 0x5C || 0x04 || 00000004 || element name relative offset || '''file-table'''
| 0x5C || 0x04 || 00000004 || element name relative offset || '''file-table'''
|- {{cellcolors|#ff8888}}
|- {{cellcolors|#ff8888}}
Line 1,394: Line 1,387:
| 0x1A0 || 0x04 || 00000000 || attribute variable 2 ||  
| 0x1A0 || 0x04 || 00000000 || attribute variable 2 ||  
|- {{cellcolors|lightgrey}}
|- {{cellcolors|lightgrey}}
! rowspan=1 | Padding
| 0x1A4 || 0x0C || 000000000000000000000000 || padding || aligned to a multiply of 0x10
| 0x1A4 || 0x0C || 000000000000000000000000 || padding || aligned to a multiply of 0x10
|-
|-
! rowspan=12 | Strings Table
! rowspan=11 | Filename Table
|- bgcolor="#88ff88"
|- bgcolor="#88ff88"
| 0x1B0 || 0x04 || 00000038 || 1st element relative offset || 0x40 + 0x38 = '''element at offset 0x78 absolute'''
| 0x1B0 || 0x04 || 00000038 || 1st element relative offset || 0x40 + 0x38 = '''element at offset 0x78 absolute'''
Line 1,419: Line 1,413:
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
|- {{cellcolors|lightgrey}}
|- {{cellcolors|lightgrey}}
! rowspan=1 | Padding
| 0x221 || 0xF || 00000000000000000000000000000000 || padding || aligned to a multiply of 0x10
| 0x221 || 0xF || 00000000000000000000000000000000 || padding || aligned to a multiply of 0x10
|-
|-
! rowspan=3 | TAGS table
! rowspan=2 | Strings table
|- bgcolor="#8888ff"
|- bgcolor="#8888ff"
| 0x230 || 0x1B || qrc file-table file src id || Container identifyer (first tag) + other tags from MAKE.XML || style="padding:0px;" | <pre style="margin:0px; padding:3px; background:#8888ff; color:#ffffff; border:none;"> <qrc>
| 0x230 || 0x1B || qrc file-table file src id || Container identifyer (first tag) + other tags from MAKE.XML || <syntaxhighlight lang="xml"> <qrc> <file-table> <file src= id=/></syntaxhighlight>
<file-table>
<file src=fileaccess id=filename/></pre>
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
|- {{cellcolors|lightgrey}}
|- {{cellcolors|lightgrey}}
! rowspan=1 | Padding
| 0x24B || 0x5 || 0000000000 || padding || aligned to a multiply of 0x10
| 0x24B || 0x5 || 0000000000 || padding || aligned to a multiply of 0x10
|- bgcolor="#ffeebb"
! Integer arrays table
| 0x250 || 0x00 ||  ||  || Empty
|- bgcolor="#ffee66"
! Float arrays table
| 0x250 || 0x00 ||  ||  || Empty
|-
|-
! rowspan="6" | Files Table
! rowspan="6" | File Table
|- bgcolor="#ffaa00"
|- bgcolor="#ffee99"
| 0x250 || 0x150 || 00001B5C... || 1st file ||  
| 0x250 || 0x150 || 00001B5C... || 1st file ||  
|- bgcolor="#ffaa00"
|- bgcolor="#ffee99"
| 0x3A0 || 0x1E0 || 00001B5B... || 2nd file ||  
| 0x3A0 || 0x1E0 || 00001B5B... || 2nd file ||  
|- bgcolor="#ffaa00"
|- bgcolor="#ffee99"
| 0x480 || 0x120 || 00001B5C... || 3rd file ||  
| 0x480 || 0x120 || 00001B5C... || 3rd file ||  
|- bgcolor="#ffaa00"
|- bgcolor="#ffee99"
| 0x5A0 || 0x1F0 || 00001B5C... || 4th file ||  
| 0x5A0 || 0x1F0 || 00001B5C... || 4th file ||  
|- bgcolor="#ffaa00"
|- bgcolor="#ffee99"
| 0x790 || 0x890 || 00001B5B... || 5th file ||  
| 0x790 || 0x890 || 00001B5B... || 5th file ||  
|- bgcolor="#888888"
|-
! Unknown area
| 0x1020 || 0x00 ||  ||  || Empty
|}
|}


'''rhm.qrc (from firmware 1.00)'''
'''rhm.qrc (from firmware 1.00)'''
<pre style="height:450px;">
<div style="height:450px; overflow:auto">
  Offset(h) 00      04      08      0C
  Offset(h) 00      04      08      0C
   
   
Line 1,714: Line 1,700:
  00001000  401F9C6C 01D0100D 8086C0C3 60409F80  @.œl.Ð..€†ÀÃ`@Ÿ€
  00001000  401F9C6C 01D0100D 8086C0C3 60409F80  @.œl.Ð..€†ÀÃ`@Ÿ€
  00001010  401F9C6C 01D0000D 8086C0C3 60411F81  @.œl.Ð..€†ÀÃ`A..  
  00001010  401F9C6C 01D0000D 8086C0C3 60411F81  @.œl.Ð..€†ÀÃ`A..  
</pre>
</div>


==P3T==
==P3T==
Line 1,790: Line 1,776:
| 0x60 || 0x04 || 00000002 || xml attribute TAG type || 2
| 0x60 || 0x04 || 00000002 || xml attribute TAG type || 2
|- {{cellcolors|#ff8888}}
|- {{cellcolors|#ff8888}}
| 0x64 || 0x04 || 3F800000 || float || 3F800000 = '''1.0'''
| 0x64 || 0x04 || 3F800000 || float ! || 3F800000
|- {{cellcolors|#ff8888}}
|- {{cellcolors|#ff8888}}
| 0x68 || 0x04 || 00000000 || Not used || 0
| 0x68 || 0x04 || 00000000 || || 0
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
| colspan="5" | element start "scene" (is the 1st children of "raf" element)
| colspan="5" | element start "scene" (is the 1st children of "raf" element)
Line 1,818: Line 1,804:
| 0x90 || 0x04 || 00000001 || integer || 1
| 0x90 || 0x04 || 00000001 || integer || 1
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0x94 || 0x04 || 00000000 || Not used || 0
| 0x94 || 0x04 || 00000000 || || 0
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
| colspan="5" | attribute start "light_num" (is an attribute of "scene" element)
| colspan="5" | attribute start "light_num" (is an attribute of "scene" element)
Line 1,828: Line 1,814:
| 0xA0 || 0x04 || 00000002 || integer || 2
| 0xA0 || 0x04 || 00000002 || integer || 2
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0xA4 || 0x04 || 00000000 || Not used || 0
| 0xA4 || 0x04 || 00000000 || || 0
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
| colspan="5" | attribute start "script_num" (is an attribute of "scene" element)
| colspan="5" | attribute start "script_num" (is an attribute of "scene" element)
Line 1,838: Line 1,824:
| 0xB0 || 0x04 || 00000001 || integer || 1
| 0xB0 || 0x04 || 00000001 || integer || 1
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0xB4 || 0x04 || 00000000 || Not used || 0
| 0xB4 || 0x04 || 00000000 || || 0
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
| colspan="5" | attribute start "actor_num" (is an attribute of "scene" element)
| colspan="5" | attribute start "actor_num" (is an attribute of "scene" element)
Line 1,848: Line 1,834:
| 0xC0 || 0x04 || 00000006 || integer || 6
| 0xC0 || 0x04 || 00000006 || integer || 6
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0xC4 || 0x04 || 00000000 || Not used || 0
| 0xC4 || 0x04 || 00000000 || || 0
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
| colspan="5" | element start "actor" (is the children of "scene" element)  
| colspan="5" | element start "actor" (is the children of "scene" element)  
Line 1,872: Line 1,858:
| 0xE8 || 0x04 || 00000005 || xml attribute TAG type || 5
| 0xE8 || 0x04 || 00000005 || xml attribute TAG type || 5
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0xEC || 0x04 || 00000000 || Float Arrays Table offset || 0
| 0xEC || 0x04 || 00000000 || || 0
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0xF0 || 0x04 || 00000004 || Array size || 4 = '''1.0,1.0,1.0,1.0'''
| 0xF0 || 0x04 || 00000004 || || 4
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
| colspan="5" | attribute start "scale" (is an attribute of "actor" element)
| colspan="5" | attribute start "scale" (is an attribute of "actor" element)
Line 1,882: Line 1,868:
| 0xF8 || 0x04 || 00000005 || xml attribute TAG type || 5
| 0xF8 || 0x04 || 00000005 || xml attribute TAG type || 5
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0xFC || 0x04 || 00000004 || Float Arrays Table offset || 4
| 0xFC || 0x04 || 00000004 || || 4
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0x100 || 0x04 || 00000003 || Array size || 3 = '''1.28,0.64,0.64'''
| 0x100 || 0x04 || 00000003 || || 3
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
| colspan="5" | attribute start "uv_offset" (is an attribute of "actor" element)
| colspan="5" | attribute start "uv_offset" (is an attribute of "actor" element)
Line 1,892: Line 1,878:
| 0x108 || 0x04 || 00000005 || xml attribute TAG type || 5
| 0x108 || 0x04 || 00000005 || xml attribute TAG type || 5
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0x10C || 0x04 || 00000007 || Float Arrays Table offset || 7
| 0x10C || 0x04 || 00000007 || || 7
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0x110 || 0x04 || 00000002 || Array size || 2 = '''0.0,0.0'''
| 0x110 || 0x04 || 00000002 || || 2
|- bgcolor="#cccccc"
|- bgcolor="#cccccc"
| colspan="5" | attribute start "color" (is an attribute of "actor" element)
| colspan="5" | attribute start "color" (is an attribute of "actor" element)
Line 1,902: Line 1,888:
| 0x118 || 0x04 || 00000005 || xml attribute TAG type || 5  
| 0x118 || 0x04 || 00000005 || xml attribute TAG type || 5  
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0x11C || 0x04 || 00000000 || Float Arrays Table offset || 0  
| 0x11C || 0x04 || 00000000 || || 0  
|- {{cellcolors|#ff9999}}
|- {{cellcolors|#ff9999}}
| 0x120 || 0x04 || 00000004 || Array size || 4 = '''1.0,1.0,1.0,1.0'''
| 0x120 || 0x04 || 00000004 || || 4  
|- bgcolor="#cccccc"
|-
| colspan="5" | attribute start "anim_weight" (is an attribute of "actor" element)
|}
|- {{cellcolors|#ff9999}}
| 0x124 || 0x04 || 00000063 || xml attribute TAG name relative offset || 63 = '''anim_weight'''
|- {{cellcolors|#ff9999}}
| 0x128 || 0x04 || 00000005 || xml attribute TAG type || 5
|- {{cellcolors|#ff9999}}
| 0x12C || 0x04 || 00000009 || Float Arrays Table offset || 9
|- {{cellcolors|#ff9999}}
| 0x130 || 0x04 || 00000004 || Array size || 4 = '''1.0,0.0,0.0,0.0'''
|- bgcolor="#cccccc"
| colspan="5" | attribute start "material" (is an attribute of "actor" element)
|- {{cellcolors|#ff9999}}
| 0x134 || 0x04 || 0000006F || xml attribute TAG name relative offset || 6F = '''material'''
|- {{cellcolors|#ff9999}}
| 0x138 || 0x04 || 00000008 || xml attribute TAG type || 8
|- {{cellcolors|#ff9999}}
| 0x13C || 0x04 || 00000000 || strings table offset || 0 = '''mtrl_logo'''
|- {{cellcolors|#ff9999}}
| 0x140 || 0x04 || 00000000 || Not used || 0
|- bgcolor="#cccccc"
| colspan="5" | attribute start "uv_scale" (is an attribute of "actor" element)
|- {{cellcolors|#ff9999}}
| 0x144 || 0x04 || 00000078 || xml attribute TAG name relative offset || 78 = '''uv_scale'''
|- {{cellcolors|#ff9999}}
| 0x148 || 0x04 || 00000005 || xml attribute TAG type || 5
|- {{cellcolors|#ff9999}}
| 0x14C || 0x04 || 0000000D || Float Arrays Table offset || D
|- {{cellcolors|#ff9999}}
| 0x150 || 0x04 || 00000002 || Array size || 2 = '''1.0,-1.0'''
|- bgcolor="#cccccc"
| colspan="5" | attribute start "id" (is an attribute of "actor" element)
|- {{cellcolors|#ff9999}}
| 0x154 || 0x04 || 00000081 || xml attribute TAG name relative offset || 6F = '''id'''
|- {{cellcolors|#ff9999}}
| 0x158 || 0x04 || 00000007 || xml attribute TAG type || 7
|- {{cellcolors|#ff9999}}
| 0x15C || 0x04 || 0000000E || strings table offset || E = '''logo'''
|- {{cellcolors|#ff9999}}
| 0x160 || 0x04 || 00000000 || Not used || 0
|-
|}
You can compare this table with the xml representation of colboot.raf contents in [http://www.psdevwiki.com/ps3/Talk:Coldboot.raf#Coldboot.sxml coldboot.raf talk page]
 
*Attribute 8 means there is another element in the xml (a different element than the one who owns the attribute) with the same name (so the loopback of his string points to a different element). In this case coldboot.raf uses the name '''mtrl_logo''' 2 times (but is only stored one time in the string table):
{{Boxcode|code=<syntaxhighlight lang="xml">
<raf> <!-- this is a reduced version of coldboot.raf.cxml -->
<scene>
<actor material="mtrl_logo"/>
</scene>
<material-table>
<material id="mtrl_logo">
</material-table>
</raf>
</syntaxhighlight>}}


=Tools=
=Tools=


==CXML decompiler==
.
Experimental python script made by flatz for research purposes, there was several versions of it where some minor changes where made, to read about that changes check the history of the page
..
 
...
The tool extracts the files inside a CXML container (CXML, QRCF, P3TF, RAFO, etc...), the extracted filenames contains the hexcodes for fileoffset and filelenght, additionally it generates an .xml that represents the original CXML structure where can be seen the order of the files and all the other data inside it
 
*Usage:
**You can copypaste the python code below and save it in your PC as cxmdecompiler.py then run it in command line without arguments to verify that it works fine (you need python v2.7 installed)
**The original files [[lines.qrc]] and [[coldboot.raf]] are zlib compressed (and the CXML decompiler cant read zlib compressed files). The command line examples below considers the files has been decompressed in a previous step (using a [[Qt_Resource_Container_(QRC)#QRC_related_tools|zlib tool]]), in that previous step the file-extension was renamed to "qrcf" and "rafo" (which are the decompressed versions where the cxml structure is fully readable and the tool can process it)
 
{{Keyboard|content=<syntaxhighlight lang="bash">C:\>cxmldecompiler.py lines.qrcf lines.xml</syntaxhighlight>}}
{{Keyboard|content=<syntaxhighlight lang="bash">C:\>cxmldecompiler.py coldboot.rafo coldboot.xml</syntaxhighlight>}}
 
'''Changelog'''
'''---------'''
v1 alpha - Basic .cxml support
v2 alpha - Fixed a bug related with RAFO header (coldboot.raf support added)
v3 alpha - Fixed offset/length displacements in the function that locates float values (xml tag attributes related with x,y,z axis in 3D space now looks correct)
v4 alpha - Floats precission increased
v5 alpha - Floats precission adjusted (previous versions had errors in the output of coldboot.raf)
v6 alpha - Now the extracted filenames contains the hexcodes for fileoffset and filelength, both values are needed for the SEARCH/REPLACE patterns to remap them in the internal .QRC structure, as can be seen in the experiments made [https://www.psx-place.com/threads/research-modifying-the-coldboot-gameboot-sequence-custom_render_plugin-sprx-rco.25952/page-19#post-214240 here] and [https://www.psx-place.com/threads/research-modifying-gaia-visualization-custom_render_plugin-earth-qrc.27756/ here]
v7 alpha - Filenames simplifyed, is better to have the filenames standarized for automated/mass comparison purposes in between official and custom files
 
{{Boxcode|height=600px|title=CXML decompiler v7 alpha (python script)|code=<syntaxhighlight lang="python">
#!python2
 
import sys, os, struct
 
from io import BytesIO
from pprint import pprint
 
def read_cstring(f):
bytes = []
while True:
byte = f.read(1)
if byte == b'\x00':
break
elif byte == '':
raise EOFError()
else:
bytes.append(byte)
return b''.join(bytes)
 
def check_file_magic(f, expected_magic):
old_offset = f.tell()
try:
magic = f.read(len(expected_magic))
except:
return False
finally:
f.seek(old_offset)
return magic == expected_magic
 
script_file_name = os.path.split(sys.argv[0])[1]
script_file_base = os.path.splitext(script_file_name)[0]
 
if len(sys.argv) < 2:
print('CXML decompiler (c) flatz')
print('Usage: {0} <cxml file> <xml file>'.format(script_file_name))
sys.exit()
 
ENDIANNESS = '>'
 
def write_raw(f, data):
if type(data) == str:
f.write(data)
elif type(data) == unicode:
f.write(data.decode('utf-8'))
else:
f.write(data)
def write_indent(f, depth):
write_raw(f, '\t' * depth)
def write_line(f, data):
write_raw(f, data)
write_raw(f, '\n')
 
INT_FMT = ENDIANNESS + 'i'
FLOAT_FMT = ENDIANNESS + 'f'
STRING_FMT = ENDIANNESS + 'ii'
INT_ARRAY_FMT = ENDIANNESS + 'ii'
FLOAT_ARRAY_FMT = ENDIANNESS + 'ii'
FILE_FMT = ENDIANNESS + 'ii'
ID_FMT = ENDIANNESS + 'i'
ID_REF_FMT = ENDIANNESS + 'i'
 
class Attribute(object):
HEADER_FMT = ENDIANNESS + 'ii'
HEADER_SIZE = struct.calcsize(HEADER_FMT)
SIZE = HEADER_SIZE + max(struct.calcsize(INT_FMT), struct.calcsize(FLOAT_FMT), struct.calcsize(STRING_FMT), struct.calcsize(INT_ARRAY_FMT), struct.calcsize(FLOAT_ARRAY_FMT), struct.calcsize(FILE_FMT), struct.calcsize(ID_FMT), struct.calcsize(ID_REF_FMT))
 
TYPE_NONE = 0
TYPE_INT = 1
TYPE_FLOAT = 2
TYPE_STRING = 3
TYPE_INT_ARRAY = 4
TYPE_FLOAT_ARRAY = 5
TYPE_FILE = 6
TYPE_ID = 7
TYPE_ID_REF = 8
 
def __init__(self, element):
self.element = element
self.start = None
self.name = None
self.type = None
self.offset = None
self.length = None
self.value = None
 
def load(self, f):
self.start = f.tell()
data = f.read(self.HEADER_SIZE)
self.name, self.type = struct.unpack(self.HEADER_FMT, data)
data = f.read(self.SIZE - self.HEADER_SIZE)
if self.type == self.TYPE_NONE:
pass
elif self.type == self.TYPE_INT:
self.value, = struct.unpack(INT_FMT, data[:struct.calcsize(INT_FMT)])
elif self.type == self.TYPE_FLOAT:
self.value, = struct.unpack(FLOAT_FMT, data[:struct.calcsize(FLOAT_FMT)])
elif self.type == self.TYPE_STRING:
self.offset, self.length = struct.unpack(STRING_FMT, data[:struct.calcsize(STRING_FMT)])
elif self.type == self.TYPE_INT_ARRAY:
self.offset, self.length = struct.unpack(INT_ARRAY_FMT, data[:struct.calcsize(INT_ARRAY_FMT)])
elif self.type == self.TYPE_FLOAT_ARRAY:
self.offset, self.length = struct.unpack(FLOAT_ARRAY_FMT, data[:struct.calcsize(FLOAT_ARRAY_FMT)])
elif self.type == self.TYPE_FILE:
self.offset, self.length = struct.unpack(FILE_FMT, data[:struct.calcsize(FILE_FMT)])
elif self.type == self.TYPE_ID:
self.offset, = struct.unpack(ID_FMT, data[:struct.calcsize(ID_FMT)])
elif self.type == self.TYPE_ID_REF:
self.offset, = struct.unpack(ID_REF_FMT, data[:struct.calcsize(ID_REF_FMT)])
return True
 
def get_name(self):
return self.element.document.get_string(self.name)
 
def get_int(self):
if self.type != self.TYPE_INT:
return None
return self.value
 
def get_float(self):
if self.type != self.TYPE_FLOAT:
return None
return self.value
 
def get_string(self):
if self.type != self.TYPE_STRING:
return None
value = self.element.document.get_string(self.offset)
if len(value) != self.length:
return None
return value
 
def get_int_array(self):
if self.type != self.TYPE_INT_ARRAY:
return None
value = self.element.document.get_int_array(self.offset, self.length)
if len(value) != self.length:
return None
return value
 
def get_float_array(self):
if self.type != self.TYPE_FLOAT_ARRAY:
return None
value = self.element.document.get_float_array(self.offset, self.length)
if len(value) != self.length:
return None
return value
 
def get_file(self):
if self.type != self.TYPE_FILE:
return None
value = self.element.document.get_file(self.offset, self.length)
return value
 
def get_id(self):
if self.type != self.TYPE_ID:
return None
id = self.element.document.get_id_string(self.offset)
return id
 
def get_id_ref(self):
if self.type != self.TYPE_ID_REF:
return None
id = self.element.document.get_id_string(self.offset)
element = Element(self.element.document)
return [id, element]
 
def dump(self, f, depth):
pass
#print('  ' * depth + 'Attribute:' + 'name:{0} type:{1}'.format(self.name, self.type), end='\n', file=f)
 
class Element(object):
HEADER_FMT = ENDIANNESS + 'iiiiiii'
SIZE = struct.calcsize(HEADER_FMT)
 
TAG_NAME = 0
ATTR_NUM = 1
PARENT = 2
PREV = 3
NEXT = 4
FIRST_CHILD = 5
LAST_CHILD = 6
 
def __init__(self, document):
self.document = document
self.start = None
self.name = None
self.num_attributes = None
self.parent = None
self.prev = None
self.next = None
self.first_child = None
self.last_child = None
 
def load(self, f):
self.start = f.tell()
self.name, self.num_attributes, self.parent, self.prev, self.next, self.first_child, self.last_child = struct.unpack(self.HEADER_FMT, f.read(self.SIZE))
return True
 
def get_name(self):
return self.document.get_string(self.name)
 
def get_attribute(self, index):
if index < 0 or index >= self.num_attributes:
return None
offset = self.start + Element.SIZE + index * Attribute.SIZE
if not is_valid_attribute(self.document, offset):
return None
attribute = Attribute(self)
f = BytesIO(self.document.tree_bin)
f.seek(offset)
attribute.load(f)
return attribute
 
def get_parent(self):
if not is_valid_element(self.document, self.parent):
return None
element = Element(self.document)
f = BytesIO(self.document.tree_bin)
f.seek(parent)
element.load(f)
return element
 
def get_first_child(self):
if not is_valid_element(self.document, self.first_child):
return None
element = Element(self.document)
f = BytesIO(self.document.tree_bin)
f.seek(self.first_child)
element.load(f)
return element
 
def get_last_child(self):
if not is_valid_element(self.document, self.last_child):
return None
element = Element(self.document)
f = BytesIO(self.document.tree_bin)
f.seek(self.last_child)
element.load(f)
return element
 
def get_prev_sibling(self):
if not is_valid_element(self.document, self.prev):
return None
element = Element(self.document)
f = BytesIO(self.document.tree_bin)
f.seek(self.prev)
element.load(f)
return element
 
def get_next_sibling(self):
if not is_valid_element(self.document, self.next):
return None
element = Element(self.document)
f = BytesIO(self.document.tree_bin)
f.seek(self.next)
element.load(f)
return element
 
def dump(self, f, depth):
write_indent(f, depth)
name = self.get_name()
write_raw(f, '<' + name)
for i in range(self.num_attributes):
attribute = self.get_attribute(i)
if attribute is None:
return False
write_raw(f, ' {0}='.format(attribute.get_name()))
if attribute.type == Attribute.TYPE_NONE:
write_raw(f, '\"null\"')
elif attribute.type == Attribute.TYPE_INT:
write_raw(f, '\"{0}\"'.format(attribute.get_int()))
elif attribute.type == Attribute.TYPE_FLOAT:
write_raw(f, '\"{0:3.7}\"'.format(attribute.get_float()))
elif attribute.type == Attribute.TYPE_STRING:
write_raw(f, '\"{0}\"'.format(attribute.get_string()))
elif attribute.type == Attribute.TYPE_INT_ARRAY:
write_raw(f, '\"')
array = attribute.get_int_array()
array_length = len(array)
for j in range(array_length):
write_raw(f, '{0:3.7}'.format(array[j]))
if j + 1 < array_length:
write_raw(f, ',')
write_raw(f, '\"')
elif attribute.type == Attribute.TYPE_FLOAT_ARRAY:
write_raw(f, '\"')
array = attribute.get_float_array()
array_length = len(array)
for j in range(array_length):
write_raw(f, '{0}'.format(array[j]))
if j + 1 < array_length:
write_raw(f, ',')
write_raw(f, '\"')
elif attribute.type == Attribute.TYPE_FILE:
file_name = 'Offset=0x{0:08X}, Length=0x{1:08X}.bin'.format(attribute.offset, attribute.length)
file_data = attribute.get_file()
with open(file_name, 'wb') as of:
of.write(file_data)
write_raw(f, '\"{0}\"'.format(file_name))
elif attribute.type == Attribute.TYPE_ID:
write_raw(f, '\"{0}\"'.format(attribute.get_id()))
elif attribute.type == Attribute.TYPE_ID_REF:
id_entity = attribute.get_id_ref()
write_raw(f, '\"{0}\"'.format(id_entity[0]))
child_element = self.get_first_child()
if not child_element is None:
write_raw(f, '>\n')
while not child_element is None:
child_element.dump(f, depth + 1)
child_element = child_element.get_next_sibling()
write_indent(f, depth)
write_raw(f, '</' + name + '>\n')
else:
write_raw(f, ' />\n')
 
def is_valid_element(document, offset):
if offset < 0 or offset + Element.SIZE > document.tree_size:
return False
element = Element(document)
f = BytesIO(document.tree_bin)
f.seek(offset)
element.load(f)
if element.num_attributes < 0 or offset + Element.SIZE + element.num_attributes * Attribute.SIZE > document.tree_size:
return False
return True
 
def is_valid_attribute(document, offset):
if offset < 0 or offset + Attribute.SIZE > document.tree_size:
return False
return True
 
class Document(object):
HEADER_FMT = ENDIANNESS + '4siiiiiiiiiiiii8x'
HEADER_SIZE = struct.calcsize(HEADER_FMT)
 
def __init__(self, file_prefix=''):
self.file_prefix = file_prefix
self.magic = None
self.version = None
self.tree_offset = None
self.tree_size = None
self.id_table_offset = None
self.id_table_size = None
self.string_table_offset = None
self.string_table_size = None
self.int_array_table_offset = None
self.int_array_table_size = None
self.float_array_table_offset = None
self.float_array_table_size = None
self.file_table_offset = None
self.file_table_size = None
self.tree_bin = None
self.id_table_bin = None
self.string_table_bin = None
self.int_array_table_bin = None
self.float_array_table_bin = None
self.file_table_bin = None
self.root = None
 
def get_document_element(self):
if not is_valid_element(self, 0):
return None
element = Element(self)
f = BytesIO(self.tree_bin)
element.load(f)
return element
 
def get_id_string(self, offset):
if offset < 0 or offset >= self.id_table_size:
return None
f = BytesIO(self.id_table_bin)
f.seek(offset)
entity_offset, = struct.unpack(INT_FMT, f.read(struct.calcsize(INT_FMT)))
return read_cstring(f)
 
def get_string(self, offset):
if offset < 0 or offset >= self.string_table_size:
return None
f = BytesIO(self.string_table_bin)
f.seek(offset)
return read_cstring(f)
 
def get_int_array(self, offset, length):
if offset < 0 or (offset + length) * struct.calcsize(INT_FMT) > self.int_array_table_size:
return None
f = BytesIO(self.int_array_table_bin)
f.seek(offset * struct.calcsize(INT_FMT))
array = []
for i in range(length):
value, = struct.unpack(INT_FMT, f.read(struct.calcsize(INT_FMT)))
array.append(value)
return array
 
def get_float_array(self, offset, length):
if offset < 0 or (offset + length) * struct.calcsize(FLOAT_FMT) > self.float_array_table_size:
return None
f = BytesIO(self.float_array_table_bin)
f.seek(offset * struct.calcsize(FLOAT_FMT))
array = []
for i in range(length):
value, = struct.unpack(FLOAT_FMT, f.read(struct.calcsize(FLOAT_FMT)))
array.append(value)
return array
 
def get_file(self, offset, length):
if offset < 0 or offset + length > self.file_table_size:
return None
return self.file_table_bin[offset:offset + length]
 
def load(self, f):
self.magic, self.version, self.tree_offset, self.tree_size, self.id_table_offset, self.id_table_size, self.string_table_offset, self.string_table_size, self.int_array_table_offset, self.int_array_table_size, self.float_array_table_offset, self.float_array_table_size, self.file_table_offset, self.file_table_size = struct.unpack(self.HEADER_FMT, f.read(self.HEADER_SIZE))
f.seek(self.tree_offset)
self.tree_bin = f.read(self.tree_size)
f.seek(self.id_table_offset)
self.id_table_bin = f.read(self.id_table_size)
f.seek(self.string_table_offset)
self.string_table_bin = f.read(self.string_table_size)
f.seek(self.int_array_table_offset)
self.int_array_table_bin = f.read(self.int_array_table_size)
f.seek(self.float_array_table_offset)
self.float_array_table_bin = f.read(self.float_array_table_size)
f.seek(self.file_table_offset)
self.file_table_bin = f.read(self.file_table_size)
self.root = self.get_document_element()
return True
 
def check(self, f):
return check_file_magic(f, 'CXML')
 
def dump(self, f=sys.stdout, depth=0):
if self.root is None:
return
self.root.dump(f, depth)
 
if len(sys.argv) < 3:
print('error: insufficient options specified')
sys.exit()
 
cxml_file_path = sys.argv[1]
if not os.path.isfile(cxml_file_path):
print('error: invalid cxml file specified')
sys.exit()
xml_file_path = sys.argv[2]
if os.path.exists(xml_file_path) and not os.path.isfile(xml_file_path):
print('error: invalid xml file specified')
sys.exit()
 
cxml_file_base = os.path.splitext(cxml_file_path)[0]
document = Document(cxml_file_base)
with open(cxml_file_path, 'rb') as f:
#if not document.check(f):
# print 'error: invalid CXML file format'
# sys.exit()
document.load(f)
 
with open(xml_file_path, 'wb') as f:
write_raw(f, '<?xml version="1.0" encoding="utf-8"?>\n')
document.dump(f)
</syntaxhighlight>}}
 
===CXML decompiler (exe for windows)===
There are tools to pack a python script together with some python libraries into an executable .exe to run it in windows. Some versions of the CXML decompiler script was distributed in this format but most probably the download inks are dead or are old versions, if someone wants to upload some of them post the links here
 
*CXML decompiler v3 alpha, download links: http://multiupload.biz/dnsipf0dkssz/cxml_decompiler_v3_alpha_MultiUpload.biz.7z.html
 




{{File Formats}}
{{File Formats}}
<noinclude>[[Category:Main]]</noinclude>
<noinclude>[[Category:Main]]</noinclude>
Please note that all contributions to PS3 Developer wiki are considered to be released under the GNU Free Documentation License 1.2 (see PS3 Developer wiki:Copyrights for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. Do not submit copyrighted work without permission!

To protect the wiki against automated edit spam, we kindly ask you to solve the following hCaptcha:

Cancel Editing help (opens in new window)

Templates used on this page:

Retrieved from "http://www.psdevwiki.com/ps3/CXML_Containers"