PlayStation archive (PSARC)
Description
PlayStation ARChive is a generic file container that uses an internal hierarchy of subfolders with defined access paths, and optional zlib or lzma per-file compression
The access paths are stored in a special file generated at build time called manifest, It mimics a standard tree structure of folders/files but the root of the hierarchy is not defined, this way the contents of the psarc can be "mounted" by the system/game as part of his filesystem at any place and under any device letter/name
There are 3 different methods to generate this paths that defines how the .psarc contents are mounted by the system/game, this methods are enabled by the archive flags stored in the header, the system/game knows how to recreate the filesystem paths and mounts the .psarc contents by reading the archive flags that are needed to process the access paths in the manifest
The compression algorithms, the PS3 system, and the .psarc format, allows for 9 compression levels but in the practise only are used level 0 (no compression) or 9 (max compression used as the default compression level)
Compression level 9 for generic file types results as a compression ratio of around 50% of the original size, the final compression ratio can vary for different types of files, usually multimedia files (video, audio, etc) doesnt admits much compression because are already compressed originally (as example, max compression ratio for .BIK video and .MP3 audio doesnt reachs 1% or 2%). For this reason is better to dont compress multimedia files because the size reduction is ridiculous and read access times can be reduced (causing a delay/lag/freeze when trying to load the file)
Very small files (smaller than 100 bytes or so) as well as dummy files (0 bytes) doesnt admits compression because are so small that the compression algorithm is not able to reduce them, they will be stored without compression inside the .psarc archive even when is applyed compression over them. When listing the contents of a .psarc with this kind of files will be displayed with a percentage of 100% and both his compressed and decompressed sizes the same value
| byte index | bit 7 | bit 6 | bit 5 | bit 4 | bit 3 | bit 2 | bit 1 | bit 0 |
|---|---|---|---|---|---|---|---|---|
| [0] CMF | Compression info
|
Compression method
| ||||||
| [1] FLG | FLEVEL
|
FDICT
(Preset dictionary) If set, a DICT dictionary identifier is present immediately after the FLG byte. The dictionary is a sequence of bytes which are initially fed to the compressor without producing any compressed output. DICT is the Adler-32 checksum of this sequence of bytes (see the definition of ADLER32 below). The decompressor can use this identifier to determine which dictionary has been used by the compressor. |
FCHECK
value must be such that CMF and FLG, when viewed as a 16-bit unsigned integer stored in MSB order (CMF *256 + FLG), is a multiple of 31. | |||||
Structure
See Discussion page
Header
PSARC header example:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 00000000 50 53 41 52 00 01 00 04 7A 6C 69 62 00 01 23 BA PSAR....zlib..#º 00000010 00 00 00 1E 00 00 09 16 00 01 00 00 00 00 00 02 ................
| Offset | Size | Name | Example | Value (conversion) | Notes |
|---|---|---|---|---|---|
| 0x00 | 0x04 | magic | 50 53 41 52 | PSAR | PlayStation ARchive |
| 0x04 | 0x04 | version | 00 01 00 04 | v1.4 | First 2 bytes is the major version, next 2 bytes is the minor version |
| 0x08 | 0x04 | compression type | 7A 6C 69 62 | zlib | zlib (default) lzma |
| 0x0C | 0x04 | toc length | 00 01 23 BA | 0x123BA | |
| 0x10 | 0x04 | toc entry size | 00 00 00 1E | 30 Bytes | Default is 30 bytes |
| 0x14 | 0x04 | toc entries | 00 00 09 16 | 1 manifest + 2325 files | The manifest is always included as the first file in the archive without an assigned ID |
| 0x18 | 0x04 | block size | 00 01 00 00 | 65536 Bytes | Default is 65536 bytes |
| 0x1C | 0x04 | archive flags | 00 00 00 02 | Absolute paths | 0 = relative paths (default) 1 = ignorecase paths 2 = absolute paths |
- Relative (default, no flags)
- Default value when the .psarc is created from command line or an .xml and no special access paths modes are specifyed
- The paths in the manifest doesnt have an slash character at the start of every line, the system accesses the files exactly like if the .psarc was a folder itself
- e.g: Folder/File.bin"
- In the games that uses .psarc's without flags (like uncharted series or the last of us) is posible to replace the psarc by a folder with the same name of the psarc and his contents decompressed inside
- Ignorecase flag
- Enabled by using the option --ignorecase in command line or ignorecase="true" in the .xml
- The paths in the manifest are case insensitive (doesnt matters wich characters are in capitals)
- Absolute flag
- Enabled by using the option --absolute in command line or absolute="true" in the .xml
- The paths in the manifest have an slash character at the start of every line,
- e.g: /Folder/File.bin"
TOC Table
Blocks info table
File table
The compression is applyed "per-block", is posible to know the compression level by looking at the zlib header
Manifest
Manifest is the first file inside the psarc filetable, doesnt appears as a file when using the --list command and doesnt have an assigned ID
In windows all the paths uses the slash "\" but when the psarc is created the paths are stored inside the manifest as "/". When making a --list command over a psarc the slashes will appear as "/"
To make things more generic all examples in wiki uses full PC paths, this allows to place the sources (files to create, or psarc to extract), targets (files to create, or psarc to extract), and psarc.exe in any path of the PC, using full paths makes needed the use of the "strip" option in all creation process to remove part of the PC path and device letter from the access paths stored in the manifest
PSARC Tools
There are several versions of PSARC related tools, features has been implemented along the different versions and is hard to keep a record of them, please help filling the info in the page
GUI versions are derivated from the command line versions
Open PSARC PS3 Extractor
By Matthieu Milan. It allows you to extract PSARC archive data in linux
Source code: https://www.ferb.fr/ps3/PSARC/
PSARC tool
Port of Matthieu Milan's open PSARC PS3 extractor to Windows. Tool written by Matthieu Milan (@usineur). Ported to VS2012 by AlexAltea.
source code: https://github.com/AlexAltea/psarc-tool
Usage: psarc [option] filename Options: -l Create a text file that lists the file id, size, and name of every file in the archive. -x Extracts all files. -e START END Extracts files with the file id specified by the range between START and END (inclusive).
PSARC Extractor
By Insomniac @ xentax forums, download link: http://www.ps3hax.net/downloads.php?do=file&id=452 and http://www.modcontrol.com/Board/pc-tools/28869-psarc-ps3-extractor.html
Usage: PSARC [flag] <fileName> Flags: -l Creates a text file that lists the File ID, Size, and Name of every file in the archive. -e START END Extracts files with the File ID specified by the range between START and END (inclusive). <blank> Extracts all files.
- Examples:
- Extract all files from data.psarc : PSARC data.psarc
- Create list of files from data.psarc : PSARC -l data.psarc
- Extract file with File ID 5 : PSARC -e 5 5 data.psarc
- Extract files with File ID 7 through 12 : PSARC -e 7 12 data.psarc
The flag -l creates a .txt file with a list of all the contents of the archive, the info included is: File ID, File original size, and File path inside psarc archive
Example of a .txt file created with the -l flag:
ID Size Name 1 | 7,39 KB | /my/files/dummies/dummy1.bin 2 | 8,80 KB | /my/files/dummies/dummy2.bin 3 | 3,30 KB | /my/files/dummies/dummy3.bin
PSARC
1.60.008 SDK : x 1.92.001 SDK : x 3.40.001 SDK : psarc-138.002 (C++), built on Feb 15 2010 at 16:17:41 3.41.001 SDK : psarc-138.002 (C++), built on Feb 15 2010 at 16:17:41 3.60.001 SDK : psarc-138.002 (C++), built on Feb 15 2010 at 16:17:41 3.70.001 SDK : psarc-138.002 (C++), built on Feb 15 2010 at 16:17:41 4.00.001 SDK : psarc-1310.001 (C++), built on Nov 16 2011 at 03:31:36 4.50.001 SDK : psarc-1310.001 (C++), built on Nov 16 2011 at 03:31:36
- From xentax forums: http://forum.xentax.com/viewtopic.php?p=44228&sid=0a7b5167ba4599f71f85471119c918a6#p44228 Download link: http://www.mediafire.com/?o3wy35uublreh
help
'''C:\>psarc.exe --help'''
'''usage:'''
psarc verb [options] [file...]
psarc --xml=XMLFILE
'''verbs:'''
create Create an archive. (default)
extract Extract contents of an existing archive.
list List contents of an existing archive.
dump Print detailed info about archive for unit testing.
verify Verify an existing archive's structural integrity and ensure that all files can be decompressed.
dtd Display a DTD describing psarc's XML input syntax.
'''general options:'''
-h, --help Show this help message and exit.
--version Display version and exit.
-d, --debug Debug: Show debug messages.
-v, --verbose Verbose: Show additional progress. (default)
-q, --quiet Quiet: Don't show any progress.
-y, --overwrite Overwrite existing files when creating/extracting.
--xml=XMLFILE XML list of actions to perform.
'''create options:'''
-oFILE, --output=FILE Archive filename to create.
-IINPUTFILE, --inputfile=INPUTFILE Inputfile listing files to archive. Consider --xml instead, which gives you more control.
-m, --mself Create a PSARC-MSELF hybrid file for PS3, which can contain encrypted special files like SDATA, SPRX, and SPU SELF.
--zlib Use Zlib when compressing. (default)
--lzma Use LZMA when compressing. (usually for PS3 PSN)
--level=N Compression quality. 1 is fastest, 9 is best. (default = 9)
-N, --nocompress Store all files uncompressed in the archive.
-bBS, --blocksize=BS Use blocks of size BS. (default = 64KiB)
-jJOBS, --jobs=JOBS Compression threads to run at a time. (default = number of CPUs)
-sREGEX, --strip=REGEX Perl-compatible regex specifying a prefix to strip from the pathnames stored in the archive. This may be specified more than once. (default = current dir)
-S, --strip-all Strip all paths from files stored in the archive.
-a, --absolute Make the paths within the archive absolute.
-R, --relative Make the paths within the archive relative. (default)
-i, --ignorecase Make the archive directory case-insensitive.
--exclude=WILDCARD Wildcard specifying files to exclude.
--skip-missing-files If set then ignore it when a file cannot be found.
--mergedups Compare the content of all files, and merge identical files so that only one copy of the data is included.
'''extract options:'''
--input=FILE Archive to extract files from. (default is first file argument)
--to=DIRECTORY Directory to write extracted files to. (default is current directory)
dtd
- Display a DTD describing psarc's XML input syntax.
The output of this command is a layout that defines the structure of an .xml document. See: http://www.w3schools.com/DTD/
'''C:\>psarc.exe dtd'''
<!ELEMENT psarc (create|extract)+>
<!ELEMENT create (compression|strip|exclude|file|directory|filelist)+>
<!ATTLIST create archive CDATA #IMPLIED
absolute (true|false) "false"
ignorecase (true|false) "false"
mergedups (true|false) "false"
stripall (true|false) "false"
blocksize CDATA "65536"
jobs CDATA #IMPLIED
skipmissingfiles (true|false) "false"
format (psarc|psarc-mself) "psarc"
overwrite (true|false) "false">
<!ELEMENT compression EMPTY>
<!ATTLIST compression type (zlib|lzma) "zlib"
level (1|2|3|4|5|6|7|8|9) "9"
enabled (true|false) "true">
<!ELEMENT strip EMPTY>
<!ATTLIST strip regex CDATA #REQUIRED>
<!ELEMENT exclude EMPTY>
<!ATTLIST exclude regex CDATA #IMPLIED
wildcard CDATA #IMPLIED>
<!ELEMENT file (CDATA)>
<!ATTLIST file wildcard CDATA #IMPLIED
regex CDATA #IMPLIED
path CDATA #IMPLIED
archivepath CDATA #IMPLIED
fromarchive CDATA #IMPLIED
compressed (true|false) #IMPLIED
skipifmissing (true|false) #IMPLIED>
<!ELEMENT filelist EMPTY>
<!ATTLIST filelist path CDATA #REQUIRED>
<!ELEMENT directory EMPTY>
<!ATTLIST directory path CDATA #REQUIRED
archivepath CDATA #IMPLIED
fromarchive CDATA #IMPLIED
compressed (true|false) #IMPLIED
skipifmissing (true|false) #IMPLIED>
<!ELEMENT extract (file)+>
<!ATTLIST extract archive CDATA #IMPLIED
input CDATA #IMPLIED
to CDATA #IMPLIED
stripall (true|false) "false"
skipmissingfiles (true|false) "false"
overwrite (true|false) "false">
- DTD Notes:
- Elements are defined in the DTD as: <!ELEMENT element-name (element-content)>
- Attributes are defined in the DTD as: <!ATTLIST element-name attribute-name attribute-type attribute-value>
- Elements that contains childrens elements that can be repeated several times are defined with a +
- Character strings attribute data types are defined with: CDATA
- Required attributes are defined with: #REQUIRED, and optional attributes with: #IMPLIED
Performing actions with an XML list
Is posible to automatize the create and extract actions by using an .xml
Using an xml is the most flexible way to use psarc.exe because has full support for all features of the tool (command line doesnt), some of the options of the tool are only available when using an xml, specially per-file attributes like: per-file compression (used by lot of games), or per-file extraction
To use an .xml is needed to prepare it based in this layout
Code Sample
- The purpose of this XML is to use it as a base layout containing all, but is not posible to use it like that, is needed to delete the tags that are not needed for a specific case, the ones that are incompatibles with other tags, and is needed to add paths, etc...
- Some attributes appears as common in the DTD for "create" and "extract" but are not common, some of them has been deleted to avoid mistakes... but probably is needed to delete some more
When the xml is ready you can run it this way:
'''C:\>psarc.exe --xml=C:\DTD_Layout.xml'''
The options --verbose, --debug, and --quiet cant be used inside the xml, so if needed are specifyed in the command line, this way:
'''C:\>psarc.exe --debug --xml=C:\DTD_Layout.xml'''
verify
- Verify an existing archive's structural integrity and ensure that all files can be decompressed.
- Output info per-file is: Validating <file ID/number>: - file path inside psarc archive - (file original size in bytes)
'''C:\>psarc.exe verify "C:\psarctests\source files\test.psarc"''' Verifying C:\psarctests\source files\test.psarc Validating 1: dummy.txt (0 bytes) Validating 2: BMPfolder1/Image1.bmp (6220854 bytes) Validating 3: JPGfolder1/JPGfolder2/Image2.jpg (12524 bytes) Validating 4: PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (6142 bytes) Archive OK
dump
- Print detailed info about archive for unit testing.
- Output info per-file is: file path inside psarc archive - file original size - file compressed size - file block start offset ?
'''C:\>psarc.exe dump "C:\psarctests\source files\test.psarc"''' Dumping C:\psarctests\source files\test.psarc dummy.txt os=0 cs=0 fb=1 BMPfolder1/Image1.bmp os=6220854 cs=8018 fb=2 JPGfolder1/JPGfolder2/Image2.jpg os=12524 cs=209 fb=97 PNGfolder1/PNGfolder2/PNGfolder3/Image3.png os=6142 cs=6142 fb=98
list
- List contents of an existing archive.
- Output info per-file is: file path inside psarc archive - (file compressed size - file original size - compression percentage)
- In the compression percentage: 100% is no reduction, 0% is a huge reduction
'''C:\>psarc.exe list "C:\psarctests\source files\test.psarc"''' Listing C:\psarctests\source files\test.psarc dummy.txt (0/0 100%) BMPfolder1/Image1.bmp (8018/6220854 0%) JPGfolder1/JPGfolder2/Image2.jpg (209/12524 1%) PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (6142/6142 100%)
'''C:\>psarc.exe list "C:\psarctests\source files\test.psarc" > c:\psarctests\filelist.txt'''
create
- Create an archive.
create by directory
Creates a psarc using the contents of a specifyed folder and all subfolders. Is only available when using an .xml because command line doesnt allows to specify a source folder
Is the most rude method to create a psarc because all folders and files will be archived alphabetically based in his names (there is no controll of the ID's that will be assigned to each file in the manifest). Cant be used to rebuild psarcs
Note in the next examle the file dummy.txt (located in the root of the folder in windows, so considered first) has been archived after BMPfolder1
Code Sample
'''C:\>psarc.exe --xml="c:\psarctests\CREATE by_directory.xml"'''
Strip: C:\psarctests\source files\test
Flags: relative
NumJobs: 2
Output: C:\psarctests\by_directory.psarc
Compression: zlib level 9
Archiving C:\psarctests\source files\test
Compressing TOC
Compressing C:\psarctests\source files\test\BMPfolder1\Image1.bmp
Compressing C:\psarctests\source files\test\dummy.txt
Compressing C:\psarctests\source files\test\JPGfolder1\JPGfolder2\Image2.jpg
Compressing C:\psarctests\source files\test\PNGfolder1\PNGfolder2\PNGfolder3\Image3.png
4: Total Files Archived
6239520: Original Size
108: Manifest (uncompressed)
72: Manifest (compressed)
380: TOC size
8836: Final Size
4.3%: Directory Overhead
99.9%: Compression Ratio
9: Compression Level
'''C:\>psarc.exe list C:\psarctests\by_directory.psarc'''
Listing C:\psarctests\by_directory.psarc
BMPfolder1/Image1.bmp (8018/6220854 0%)
dummy.txt (0/0 100%)
JPGfolder1/JPGfolder2/Image2.jpg (209/12524 1%)
PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (157/6142 2%)
create by filelist
This method allows to order the files inside the archive in a specific position (so is posible to know the ID that will have every files before creating the psarc)
When rebuilding a psarc from a game is needed to preserve the original ID's of ALL the files inside the archive (is posible to replace files but never remove). By replacing files (e.g: by dummyes) all the others preserves his positions
The filelist is a plain .txt file where every file is defined by a single line, composed by his path, filename, and file extension, 1 line per file. Is posible to generate this kind of filelists automatically by using the output of the --list command and some regular expressions to replace characters
The examples used in wiki uses full PC paths to be more flexible (no need to place psarc.exe next to the source folder, actually you can place psarc.exe in c:\ the source folder in d:\ and create the psarc in x:\). When using full paths is mandatory to use the --strip option with an explicit path or a regular expression (regex) to remove part of the PC path (e.g: if you strip C:\psarctests\source files\test from C:\psarctests\source files\test\BMPfolder1\Image1.bmp the resulting path of the file inside the archive will be BMPfolder1/Image1.bmp)
The --strip option removes part of the paths from the filelist, can be used several times with different stripped paths, this is usefull when the psarc archive is generated from a filelist that contains paths from different folders and/or devices in the PC (e.g: is posible to create a filelist including a file from F:\textures\packed\texture1.jpg and another file from G:\models\packed\model1.xxx and strip the paths: F:\textures\ and G:\models\, this way both files will be stored inside the psarc archive together under path: packed/texture1.jpg and packed/model1.xxx)
First is needed to prepare a filelist.txt like this example, located in C:\psarctests\source files\filelist.txt, contains the PC paths of the files that are in the folder C:\psarctests\source files\test and will be included inside the psarc archive:
C:\psarctests\source files\test\dummy.txt C:\psarctests\source files\test\BMPfolder1\Image1.bmp C:\psarctests\source files\test\JPGfolder1\JPGfolder2\Image2.jpg C:\psarctests\source files\test\PNGfolder1\PNGfolder2\PNGfolder3\Image3.png
The psarc archive is created by using the filelist as the --inputfile and uses an explicit path to --strip from every file inside the filelist, this way:
'''C:\>psarc.exe create --strip="C:\psarctests\source files\test" --output=C:\psarctests\by_filelist.psarc --inputfile="C:\psarctests\source files\filelist.txt"'''
Or you can prepare an .xml that contains the path to the filelist.txt
Code Sample
Then run the contents of the .xml this way:
'''C:\>psarc.exe --xml="c:\psarctests\CREATE by_filelist.xml"'''
The output of the tool is the same in both cases (except the InputFile not present when using an xml)
Strip: C:\psarctests\source files\test
Flags: relative
NumJobs: 2
InputFile: C:\psarctests\source files\filelist.txt
Output: C:\psarctests\by_filelist.psarc
Compression: zlib level 9
Archiving C:\psarctests\source files\test\dummy.txt
Archiving C:\psarctests\source files\test\BMPfolder1\Image1.bmp
Archiving C:\psarctests\source files\test\JPGfolder1\JPGfolder2\Image2.jpg
Archiving C:\psarctests\source files\test\PNGfolder1\PNGfolder2\PNGfolder3\Image3.png
Compressing TOC
Compressing C:\psarctests\source files\test\dummy.txt
Compressing C:\psarctests\source files\test\BMPfolder1\Image1.bmp
Compressing C:\psarctests\source files\test\JPGfolder1\JPGfolder2\Image2.jpg
Compressing C:\psarctests\source files\test\PNGfolder1\PNGfolder2\PNGfolder3\Image3.png
4: Total Files Archived
6239520: Original Size
108: Manifest (uncompressed)
72: Manifest (compressed)
380: TOC size
8836: Final Size
4.3%: Directory Overhead
99.9%: Compression Ratio
9: Compression Level
'''C:\>psarc.exe list C:\psarctests\by_filelist.psarc'''
Listing C:\psarctests\by_filelist.psarc
dummy.txt (0/0 100%)
BMPfolder1/Image1.bmp (8018/6220854 0%)
JPGfolder1/JPGfolder2/Image2.jpg (209/12524 1%)
PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (157/6142 2%)
create by file
This method is the real deal, only available by using an xml, superceedes all the other creation methods and is the only method where is posible to use per-file attributes
The xml includes a list of files where every path of every file in the PC is specifyed (paths are the same than the filelist.txt explained before) but every file line allows for attributes
Note in the next example the compression attribute is enabled and affects all files, except Image3.png that has been archived without compression (appears as 100% in the list command output)
The other per-file attribute avoids the tool to output an error in the case the file dummy.txt is missing in the source folder
Code Sample
C:\>psarc.exe --xml="c:\psarctests\CREATE by_file.xml"
Strip: C:\psarctests\source files\test
Flags: relative
NumJobs: 2
Output: C:\psarctests\by_file.psarc
Compression: zlib level 9
Archiving C:\psarctests\source files\test\dummy.txt
Archiving C:\psarctests\source files\test\BMPfolder1\Image1.bmp
Archiving C:\psarctests\source files\test\JPGfolder1\JPGfolder2\Image2.jpg
Archiving C:\psarctests\source files\test\PNGfolder1\PNGfolder2\PNGfolder3\Image3.png
Compressing TOC
Compressing C:\psarctests\source files\test\dummy.txt
Compressing C:\psarctests\source files\test\BMPfolder1\Image1.bmp
Compressing C:\psarctests\source files\test\JPGfolder1\JPGfolder2\Image2.jpg
Storing C:\psarctests\source files\test\PNGfolder1\PNGfolder2\PNGfolder3\Image3.png
4: Total Files Archived
6239520: Original Size
108: Manifest (uncompressed)
72: Manifest (compressed)
380: TOC size
14821: Final Size
2.6%: Directory Overhead
99.8%: Compression Ratio
9: Compression Level
'''C:\>psarc.exe list C:\psarctests\by_file.psarc'''
Listing C:\psarctests\by_file.psarc
dummy.txt (0/0 100%)
BMPfolder1/Image1.bmp (8018/6220854 0%)
JPGfolder1/JPGfolder2/Image2.jpg (209/12524 1%)
PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (6142/6142 100%)
Batch. This is an useless example to show how the .psarcs are created sequentially and how to create several .psarc's with only one .xml, it uses all default values for compressions, formats, flags etc...
In step 1 it creates an egg.psarc containing a dummy.txt. In step 2 it creates a chiken.psarc containing the egg.psarc made in step 1. In step 3 it creates an egg.psarc containing the chicken.psarc made in step 2 (egg.psarc created in step 3 overwrites egg.psarc made in step 1)
Code Sample
'''C:\>psarc.exe --xml="c:\psarctests\CREATE by_file_paradox.xml"'''
Strip: C:\psarctests\source files\test
Flags: relative
NumJobs: 2
Output: C:\psarctests\egg.psarc
Compression: zlib level 9
Archiving C:\psarctests\source files\test\dummy.txt
Compressing TOC
Compressing C:\psarctests\source files\test\dummy.txt
1: Total Files Archived
0: Original Size
9: Manifest (uncompressed)
9: Manifest (compressed)
96: TOC size
105: Final Size
91.4%: Directory Overhead
-1.$%: Compression Ratio
9: Compression Level
Strip: C:\psarctests
Flags: relative
NumJobs: 2
Output: C:\psarctests\chicken.psarc
Compression: zlib level 9
Archiving C:\psarctests\egg.psarc
Compressing TOC
Compressing C:\psarctests\egg.psarc
1: Total Files Archived
105: Original Size
9: Manifest (uncompressed)
9: Manifest (compressed)
96: TOC size
174: Final Size
55.2%: Directory Overhead
-65.7%: Compression Ratio
9: Compression Level
Strip: C:\psarctests
Flags: relative
NumJobs: 2
Output: C:\psarctests\egg.psarc
Compression: zlib level 9
Archiving C:\psarctests\chicken.psarc
Compressing TOC
Compressing C:\psarctests\chicken.psarc
1: Total Files Archived
174: Original Size
13: Manifest (uncompressed)
26: Manifest (compressed)
96: TOC size
252: Final Size
38.1%: Directory Overhead
-44.8%: Compression Ratio
9: Compression Level
'''C:\>psarc.exe list C:\psarctests\egg.psarc
Listing C:\psarctests\egg.psarc
chicken.psarc (143/174 82%)
'''C:\>psarc.exe list C:\psarctests\chicken.psarc
Listing C:\psarctests\chicken.psarc
egg.psarc (69/105 65%)
extract
- Extract contents of an existing archive.
- There are 3 optional available attributes for extract:
- --overwrite (if the files existed in the extraction folder they will be overwritten without advise)
- --strip-all (extracts all files in the root of the target folder, incase of several extracted files with the same name the program will return an error, this can be solved by using the command --overwrite)
- --skip-missing-files (seems pointless, mostly to extract files from malformed .psarcs or .psarcs that doesnt passes the "verify" command ?)
The required options are --input (the source .psarc) and --to (the target folder where contents will be extracted)
extract all
The example command extracts all files from the .psarc to a directory with full directory tree names, no skipped missing files, no overwrite
When using this options the command line output is the same, there is no mention of wich files are stripped, overwritten, or missing in the extraction process
'''C:\>psarc.exe extract --input="C:\psarctests\source files\test.psarc" --to="C:\psarctests\test"''' Extracting: dummy.txt (0 bytes) Extracting: BMPfolder1/Image1.bmp (6220854 bytes) Extracting: JPGfolder1/JPGfolder2/Image2.jpg (12524 bytes) Extracting: PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (6142 bytes) '''C:\>psarc.exe extract --overwrite --strip-all --skip-missing-files --input="C:\psarctests\source files\test.psarc" --to="C:\psarctests\test"''' Extracting: dummy.txt (0 bytes) Extracting: BMPfolder1/Image1.bmp (6220854 bytes) Extracting: JPGfolder1/JPGfolder2/Image2.jpg (12524 bytes) Extracting: PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (6142 bytes)
extract all (xml)
Code Sample
'''C:\>psarc.exe --xml="c:\psarctests\EXTRACT all.xml"''' Extracting: dummy.txt (0 bytes) Extracting: BMPfolder1/Image1.bmp (6220854 bytes) Extracting: JPGfolder1/JPGfolder2/Image2.jpg (12524 bytes) Extracting: PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (6142 bytes)
extract by file
This extraction method is only available when using an xml because command line doesnt allows to specify files inside the archive
Code Sample
'''C:\>psarc.exe --xml="c:\psarctests\EXTRACT by_file.xml"''' Extracting: BMPfolder1/Image1.bmp (6220854 bytes)
When extracting single files is also posible to use the same attributes availables in command line, and attributes per-file, but they uses different names inside the xml (see the DTD example)
Code Sample
- Notes about the xml examples above:
- The element "extract" uses the attribute archive but could be replaced by input, in this case are exactly the same and both will work, actually --input is the same option used in comand line but the examples in wiki uses the name archive because is more intuitive (is also used as an attribute for "create" and in both cases is refered to the compressed archive.psarc). Both (archive and input) appears in the DTD as optional (#IMPLIED) but only one of them can be used in the xml
- The element "file" uses the attribute archivepath but could be replaced by path, in this case are exactly the same and both will work, but the examples in wiki uses the name archivepath because is more intuitive (is refered to the path of a file inside archive.psarc). Both (archivepath and path) appears in the DTD as optional (#IMPLIED) but only one of them can be used in the xml
Total Commander plugin
http://www.totalcmd.net/plugring/PSARC.html
PS3 PSARC GUI
It's a graphical user interface to view, extract or create psarc archives. GUI written by aldo, unknown psarc tool used internally
Download link: http://aldostools.org/ps3_psarc_gui.rar
Rebuilding PSARC files
When rebuilding a psarc is needed to preserve the original positions of all the files inside the archive (because every file in the original psarc has assigned an ID number, and after rebuilding this numbers must match with the original positions). for this same reason when rebuilding is not a good idea to remove files (because the other files that comes after will be displaced). When the purpose of the rebuild is to reduce the size (by removing languages, etc...) the method consists in replacing them by dummies (0kb files), this dummies preserves the positions of all the files
This need to preserve the original positions of all the files inside the archive in the practise means there are only 2 methods to rebuild a psarc: "by filelist" or "by file". For both is needed to create a list of files
- Step 1 - Identify compression method, archive flags, and verify default structure sizes
- Step 2 - Identify if per-file compression was used
- Step 3a - If per-file compression was not used, rebuild with the method "by filelist" or "by file"
- Step 3a - If per-file compression was used, rebuild with the method "by file"
Step 1
Is needed to look at the psarc header to identify some settings
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 00000000 50 53 41 52 00 01 00 04 7A 6C 69 62 00 01 23 BA PSAR....zlib..#º 00000010 00 00 00 1E 00 00 09 16 00 01 00 00 00 00 00 03 ................
Most important ones are the compression method (zlib/lzma) and the archive flags (relative/absolute, and ignorecase), this settings will be used when rebuilding from command line or from xml this way:
Code Sample
The other values in the header that needs to be verifyed are the TOC entry size and the Block size, is needed to know if the original psarc we are trying to rebuild uses the default values (all i found used defaults), otherway is posible to use non-default values this way:
Code Sample
Step2
Now is needed to know if the psarc uses per-file compression (if some of the files inside the archive ara compressed, and others are not compressed)
This is important to know wich rebuild method will be used in the next step, is needed to make a --list command with psarc tool and store the output of the command as a .txt
Open the .txt in a text editor (like notepad++) and search for the string 100% (the files that appears with a 100% has been archived but are not compressed). Is needed to do it this way semi automatized because is usual to find psarc with thousand of files inside
- Check how many files are not compressed, his sizes, and file extensions
- If there are no files with 100% or if there are only a few and his sizes are minimal this is normal, it means the archive doesnt uses per-file compression
- If there some files with the 100%, normal sizes (not minimal), using the same file extension, and can be identifyed (like audio or video formats) then you have a psarc with per-file compression
- If all the files appears as 100% it means the psarc has been created with the compression attibute set as "false" this way:
Code Sample
Step3
If the psarc doesnt uses per-file compression then is posible to rebuild it with the create by filelist method from command line by using an external filelist.txt
If the psarc uses per-file compression then the only way to rebuild it is with the create by file method by using an xml (the xml includes a list of files like the filelist.txt)
- Special rebuild notes:
- Mass effect 3 MAIN.PSARC contains a group of generic files compressed with level 9, and another group of .TXT, .BIN, and .BIK videos not compressed at all. In the creation options suggested above in the table the compression level is not defined and this means all them will use the default compression level 9 (so the .BIK files will have a ridiculous 1% or 2% of compression ratio because is the max they will admit). This is not how the original file was created (but works), the correct way to rebuild it is by defining specific compression levels "per file" in a predefined .xml
- Killzone 3 data_ps3.psarc (BCES01007 version) contains 8324 files (included 5.3GB of audio files for 21 languages with a compression of around 50%)... 2883 of all the files uses the same path/filenames repeated up to 3 times (because was created using 3 or more different working directories). This duplication of paths/filenames inside the .psarc in the practise means when doing a standard extraction 2833 files will be missing, overwrited, or they will give an error at extraction time, the correct way to mess with it is by doing several partiall extractions to different directories, and rebuild it by using a preconfigured .xml where the different paths for each file are specifyed, when rebuilding is also needed to use the --strip option several times to remove part of the paths from all the different working directories, this way the paths/filenames will be duplicated inside the .psarc archive (but are different files located at different positions and with a different ID number assigned).
- Ape Escape target.psarc includes "dupes" (or duplicated files) in the TOC (table of contents) of the psarc archive. This method used at creation time basically works by making several entries of the TOC to point to the same file, this way a single file inside the psarc is indexed several times in the TOC and can be accessed from several different paths/filenames. The only way to identify this dupes before extraction is by reading the TOC of the psarc for all the files and compare wich position are pointing to. As an example the original size of target.psarc is 328mb... after a "standard" extraction is 2.52GB (this extraction doesnt identify the dupes so are extracted several times under different paths/filenames)... if you try to rebuild it again his final size will be much bigger than the original target.psarc
Generating a filelist.txt or xml by using regular expressions
- The file paths inside filelist.txt or inside an xml admits 2 formats:
- full PC paths (where the full PC file path included the device letter is used by psarc.exe to locate the file in any device and/or any folder). This method is the most flexible but needs an strip command to remove part of the PC path when creating the psarc archive
- relative to work directory paths (where work directory is the path to psarc.exe and the files needs to be placed in a subfolder under it). This method needs to copy psarc.exe to the parent folder of the files
The file list we need to generate contains the access paths stored in the manifest, the easy way to have have a copy of the manifest is by using the list command of psarc tool
The output of the list command has some characters that are needed ro remove or replace, the purpose of automatizing this process by using regular expressions is because usually psarc files contains thousand of files, the examples used here are commands for fnr.exe (find and replace tool) and contains regular expressions (to search) and a character strings (to replace)
Generating a filelist
First generate a .txt file containing the output of the list comand
'''C:\>psarc.exe list "C:\psarctests\source files\test.psarc" > "C:\psarctests\filelist.txt"'''
The file generated by the list command will have this content (note there is en empty line at the end):
Listing C:\psarctests\source files\test.psarc dummy.txt (0/0 100%) BMPfolder1/Image1.bmp (8018/6220854 0%) JPGfolder1/JPGfolder2/Image2.jpg (209/12524 1%) PNGfolder1/PNGfolder2/PNGfolder3/Image3.png (6142/6142 100%)
- Replacing slashes (this step is not really needed, both slashes are valid, but is good to do it to standarize all paths formats, otherway is a mess)
find:
/
replace by:
\\
The resulting file:
Listing C:\psarctests\source files\test.psarc dummy.txt (0\0 100%) BMPfolder1\Image1.bmp (8018\6220854 0%) JPGfolder1\JPGfolder2\Image2.jpg (209\12524 1%) PNGfolder1\PNGfolder2\PNGfolder3\Image3.png (6142\6142 100%)
- Removing first line
find:
Listing.*\n
replace by:
sourcepath
The resulting file:
sourcepathdummy.txt (0\0 100%) BMPfolder1\Image1.bmp (8018\6220854 0%) JPGfolder1\JPGfolder2\Image2.jpg (209\12524 1%) PNGfolder1\PNGfolder2\PNGfolder3\Image3.png (6142\6142 100%)
- Removing parenthesis
find:
.\(.*\n
replace by:
\r\nsourcepath
The resulting file:
sourcepathdummy.txt sourcepathBMPfolder1\Image1.bmp sourcepathJPGfolder1\JPGfolder2\Image2.jpg sourcepathPNGfolder1\PNGfolder2\PNGfolder3\Image3.png sourcepath
- Removing last line
find:
\r\nsourcepath$
replace by: nothing, the resulting file:
sourcepathdummy.txt sourcepathBMPfolder1\Image1.bmp sourcepathJPGfolder1\JPGfolder2\Image2.jpg sourcepathPNGfolder1\PNGfolder2\PNGfolder3\Image3.png
- Replacing sourcepath by a real path (this step was not really needed because you could use the real path in the steps before, but i prefer to make the change in this last step because all the previous ones are generic)
Now you can replace the word "sourcepath" we have been using as a placeholder by the folder path where the psarc contents are decompressed, e.g: C:\psarctests\source files\test\, remember this is exactly the part of the path that is needed to strip later (because we dont want to include our local path C:\psarctests\source files\test\ inside the manifest)
find:
sourcepath
replace by:
C:\\psarctests\\source files\\test\\
The resulting file:
C:\psarctests\source files\test\dummy.txt C:\psarctests\source files\test\BMPfolder1\Image1.bmp C:\psarctests\source files\test\JPGfolder1\JPGfolder2\Image2.jpg C:\psarctests\source files\test\PNGfolder1\PNGfolder2\PNGfolder3\Image3.png
And thats all, this is a perfect filelist.txt that can be used for the rebuild method by filelist (usefull when the psarc doesnt uses per-file compression)
Generating a XML
The way to create an xml is pretty similar, first generate a .xml file containing the output of the list comand
'''C:\>psarc.exe list "C:\psarctests\source files\test.psarc" > "C:\psarctests\filelist.xml"'''
The file generated by the list command will have this content (note there is en empty line at the end):
Code Sample
- Replacing slashes (this step is not really needed, both slashes are valid, but is good to do it to standarize all paths formats, otherway is a mess)
find:
/
replace by:
\\
The resulting file:
Code Sample
- Replacing first line
find:
Listing.*\n
replace by:
<psarc> <create archive="targetpsarc" absolute="false" ignorecase="false"> <compression type="zlib" enabled="true" /> <strip regex="sourcepath" /> <file path="sourcepath
The resulting file:
Code Sample
- Replacing parenthesis
find:
.\(.*\n
replace by:
" /> <file path="sourcepath
The resulting file:
Code Sample
- Replacing last line
find:
<file path="sourcepath$
replace by:
</create> </psarc>
The resulting file:
Code Sample
- Replacing path placeholders
This step was not really needed because you could use the real path in the steps before, but i prefer to make the change in this last step because all the previous ones are generic Now you can replace the word "sourcepath" we have been using as a placeholder by the folder path where the psarc contents are decompressed, e.g: C:\psarctests\source files\test\, also replace the path "targetpsarc" where the psarc will be created, e.g: C:\psarctests\test.psarc
find:
sourcepath
replace by:
C:\\psarctests\\source files\\test\\
find:
targetpsarc
replace by:
C:\\psarctests\\test.psarc
The resulting file:
Code Sample
Now is needed to verify all the common settings are fine (ive used default values for the replacements in the examples)
- Adding per-file attributes (optional)
If the psarc uses per-file attributes now is needed to apply them by identifying filenames or file extensions, e.g: if we know the png images doesnt have compression it can be automatized this way:
find:
.png" />
replace by:
.png" compressed="false" />
The resulting file:
Code Sample
Games or Apps using PSARC files
| ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Game/App info | PSARC Header info | Compression Method |
Notes | |||||||||
| TITLE | PSARC files | Version (offset 0x04) |
Compression (offset 0x08) |
TOC entry size (offset 0x10) |
Block Size (offset 0x18) |
Flags (offset 0x1C) | ||||||
| Ape Escape | sound.psarc | 00010004 | zlib | 0000001E | 00010000 | 00000000 (no flags) | no compress | Contains not-compressed audio | ||||
| <language>.psarc | 00010004 | zlib | 0000001E | 00010000 | 00000002 (absolute) | compress all | ||||||
| target.psarc | 00010004 | zlib | 0000001E | 00010000 | 00000002 (absolute) | compress all | TOC dupes!!!, see notes below | |||||
| Atelier Rorona Plus | ? | 00010004 | zlib | 0000001E | 00010000 | 00000002 (absolute) | ? | |||||
| Eiyuu Densetsu Sen no Kiseki | data.psarc | |||||||||||
| Eiyuu Densetsu Sen no Kiseki II | data.psarc | |||||||||||
| God of War III | All | 00010004 | zlib | 0000001E | 00010000 | 00000002 (absolute) | compress all | |||||
| God of War Ascension | INTROCIN.PSARC | no compress | Intro video | |||||||||
| CRTN.PSARC CYN.PSARC FURY.PSARC HECA.PSARC HRTN.PSARC LAN.PSARC PHI.PSARC STAT.PSARC STINT.PSARC TEM.PSARC |
per-file compress | Contains not-compressed audio, and other compressed files game levels, and game levels soundtracks | ||||||||||
| RESOURCES.PSARC | per-file compress | Game resources, included not-compresed audio | ||||||||||
| <LANGUAGE>.PSARC | per-file compress | Contains not-compressed audio, and other compressed files | ||||||||||
| SHELL.PSARC TPPS.PSARC |
compress all | Main menu Game updates stub (dummy file) | ||||||||||
| GAME.PSARC (or all others ?) |
00010004 | zlib | 0000001E | 00010000 | 00000002 (absolute) | compress all | Game settings | |||||
| ICO & Shadow of the Colossus HD | ||||||||||||
| Killzone 3 | data_ps3.psarc | 00010004 | zlib | 0000001E | 00010000 | 00000001 (ignorecase) | compress all | created from 3 working directories, lot of duplicated paths | ||||
| Mass Effect 3 | AUDIO.PSARC | 00010004 | lzma | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | no compress | Contains not-compressed audio | ||||
| MAIN.PSARC | 00010004 | lzma | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | per-file compress | Contains not-compressed video, and other compressed files | |||||
| TEXTURES.PSARC | ||||||||||||
| Motorstorm Pacific Rift | ? | |||||||||||
| Ni No Kuni | bd03.adat ? hd01.adat ? hd05<language>.adat ? |
.psarc file extensions renamed to .adat | ||||||||||
| Ninja Gaiden 3 Razors Edge | ||||||||||||
| Ratchet & Clank. Tools of Destruction | *_uncached.psarc | 00010002 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | per-file compress | Contains not-compressed audio, and other compressed files | ||||
| All others | 00010002 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | compress all | ||||||
| Ratchet & Clank. Quest for Booty | *_uncached.psarc | 00010002 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | per-file compress | Contains not-compressed audio, and other compressed files | ||||
| All others | 00010002 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | compress all | global_cached.psarc contains TOC dupes!!! (icon0.png) | |||||
| Ratchet & Clank. A Crack in Time | *_uncached.psarc | 00010003 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | per-file compress | Contains not-compressed audio, and other compressed files | ||||
| All others | 00010003 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | compress all | ||||||
| Ratchet & Clank. Into the Nexus | global_cached.psarc level_cached.psarc |
00010003 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | per-file compress | Contains not-compressed files, and other compressed files | ||||
| global_uncached.psarc global_sound_**.psarc |
00010003 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | per-file compress | Contains not-compressed audio, and other compressed files | |||||
| All others | 00010003 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | compress all | ||||||
| The Last Of Us | All ? | 00010003 | zlib | 0000001E | 00010000 | 00000000 (no flags) | ||||||
| The Shoot | All | 00010004 | zlib | 0000001E | 00010000 | 00000002 (absolute) | ||||||
| Time and Eternity | ||||||||||||
| Uncharted 2 | All ? | 00010003 | zlib | 0000001E | 00010000 | 00000000 (no flags) | ||||||
| Uncharted 3 | All | 00010003 | zlib | 0000001E | 00010000 | 00000000 (no flags) | ||||||
| Warhawk | ? | |||||||||||
| White Knight Chronicles II | All .psarc | 00010003 | zlib | 0000001E | 00010000 | 00000002 (absolute) | no compress | Contains not-compressed audio | ||||
| All .psarc.sdat | 00010003 | zlib | 0000001E | 00010000 | 00000002 (absolute) | compress all | Default compression, NPD signed psarcs | |||||
| Wipeout HD Fury | data01.psarc | 00010003 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | no compress | Contains not-compressed audio | ||||
| All others | 00010003 | zlib | 0000001E | 00010000 | 00000003 (ignorecase + absolute) | compress all | ||||||
| ||||||||||||
| Game/App info | PSARC Header info | Compression Method |
Notes | |||||||||
| TITLE | PSARC files | Version (offset 0x04) |
Compression (offset 0x08) |
TOC entry size (offset 0x10) |
Block Size (offset 0x18) |
Flags (offset 0x1C) | ||||||
| VidZone | vidzone*.db.psarc | 00010004 | zlib | 0000001E | 00010000 | 00000000 (no flags) | compress all | Vidzone is a web service, the .psarc is hosted in the web | ||||
- Other games that probably uses PSARC's
- Uncharted 1?
- Killzone colletion ?
- God of War. Collection ?
- other Motorstorms ?
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
