PFS: Difference between revisions

From PS4 Developer wiki
Jump to navigation Jump to search
mNo edit summary
No edit summary
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''PFS''' (Playstation File System) is the file system used by (at least) downloadable content and games on the PS4. It is loosely based on the [[wikipedia:UFS|UFS]] (Unix File System) used in FreeBSD. PFS typically uses a 64kB block size, although the block size is configurable and specified in the file system header. The minimum block size is 4kB, and the maximum seems to be 16MiB; the block size must be a power of 2.
'''PFS''' (Pseudo/Playstation File System) is the file system used by (at least) downloadable content and games on the PS4. It is loosely based on the [[wikipedia:UFS|UFS]] (Unix File System) used in FreeBSD. PFS typically uses a 64kB block size, although the block size is configurable and specified in the file system header. The minimum block size is 4kB, and the maximum is 32MiB; the block size must be a power of 2.


== Structure ==
= Structure =


There are four main sections in a Playstation File System:
There are four main sections in PFS:
* Header (superblock)
* Header (superblock)
* Inode blocks
* Inode blocks
Line 9: Line 9:
* Data blocks
* Data blocks


=== Header/Superblock ===
== Header/Superblock ==
 
{| class="wikitable"
{| class="wikitable"
! Offset !! Value !! Size !! Notes
! Offset !! Value !! Size !! Notes
|-
|-
| 0x00 || version || 0x8 || Should be 1
| 0x00 || version || 0x8 || Always 1
|-
|-
| 0x08 || magic || 0x8 || Should be 20130315. Used like a version number for the PFS.
| 0x08 || format || 0x8 || Always 20130315
|-
|-
| 0x10 || id || 0x8 ||  
| 0x10 || id || 0x8 ||  
Line 31: Line 32:
| 0x1E || unknown || 0x2 ||  
| 0x1E || unknown || 0x2 ||  
|-
|-
| 0x20 || blocksz || 0x4 || The size of each block in the filesystem.
| 0x20 || blocksz || 0x4 || The size of each block in the filesystem
|-
|-
| 0x24 || nbackup || 0x4 ||  
| 0x24 || nbackup || 0x4 || Seems to always be 0
|-
|-
| 0x28 || nblock || 0x8 ||  
| 0x28 || nblock || 0x8 || Seems to always be 1
|-
|-
| 0x30 || ndinode || 0x8 || Number of inodes in the inode blocks
| 0x30 || ndinode || 0x8 || Number of inodes in the inode blocks
Line 45: Line 46:
| 0x48 || superroot_ino || 0x8 ||  
| 0x48 || superroot_ino || 0x8 ||  
|}
|}
<source lang="c">
<source lang="c">
typedef struct {
typedef struct {
Line 66: Line 68:
</source>
</source>


=== Inodes ===
== Inodes ==
 
Inode table starts at the second block and continues for <tt>header.ndinodeblock</tt> blocks.
Inode table starts at the second block and continues for <tt>header.ndinodeblock</tt> blocks.
There are only <tt>header.blocksz / sizeof(di_d32)</tt> inodes per block. (inodes will never cross a block boundary)
There are only <tt>header.blocksz / sizeof(di_d32)</tt> inodes per block. (inodes will never cross a block boundary)


For an explanation of inodes, direct and indirect blocks, see [[wikipedia:Inode_pointer_structure|inode pointer structure]] on Wikipedia. Most PFS images from PKGs have just one direct block pointer per file, and the file's data just takes up consecutive blocks. But if there is fragmentation on the file system, you will have to follow the direct and indirect block pointers.
For an explanation of inodes, direct and indirect blocks, see [[wikipedia:Inode_pointer_structure|inode pointer structure]] on Wikipedia. Most PFS images from PKGs have just one direct block pointer per file, and the file's data just takes up consecutive blocks. But if there is fragmentation on the file system, you will have to follow the direct and indirect block pointers.
There are actually 4 types of inodes that can be used in a PFS image, and the type is determined by the low two bits of the<tt>mode</tt> field in the superblock. Most PFS images that you see, such as pfs_image.dat, will be using mode 0, which refers to non-signed 32-bit inodes.
{| class="wikitable"
{| class="wikitable"
! Offset !! Value !! Size !! Notes
! Offset !! Value !! Size !! Notes
|-
|-
| 0x00 || mode|| 0x2 || Inode mode (bitwise OR of flags; file=0x8000, directory=0x4000)
| 0x00 || mode|| 0x2 || Inode mode (bitwise OR of flags; file=0x8000, directory=0x4000), low 9 bits are file permissions
|-
|-
| 0x02 || nlink || 0x2 ||  
| 0x02 || nlink || 0x2 || Number of links to this inode. For files, this is 1. For dirs, this is 1 plus the number of subdirectories (the <tt>..</tt> entry links to the subdir's parent, increasing its <tt>nlink</tt> count).
|-
|-
| 0x04 || flags|| 0x4 ||  
| 0x04 || flags|| 0x4 || Bitfield of flags for compressed, readonly, etc.
|-
|-
| 0x08 || size || 0x8 ||  
| 0x08 || size || 0x8 || Size in bytes of the entity
|-
|-
| 0x10 || unknown || 0x38 ||  
| 0x10 || size_compressed || 0x8 || same as <tt>size</tt> for uncompressed PFS
|-
|-
| 0x48 || uid|| 0x4 ||  
| 0x18 || times|| 0x30 || Four 64-bit unix timestamps, followed by four 32-bit zeroes, which may be nanoseconds.
|-
|-
| 0x4C || gid|| 0x4 ||  
| 0x48 || uid|| 0x4 || User ID (zero for game PFS images at least)
|-
|-
| 0x50 || unknown || 0x10 ||  
| 0x4C || gid|| 0x4 || Group ID (zero for game PFS images at least)
|-
| 0x50 || spare || 0x10 || Probably the same as di_spare, always 0
|-
|-
| 0x60 || blocks || 0x4 || Number of blocks occupied
| 0x60 || blocks || 0x4 || Number of blocks occupied
Line 96: Line 104:
| 0x94 || ib || 0x14 || Indirect blocks
| 0x94 || ib || 0x14 || Indirect blocks
|}
|}
<source lang="c">
<source lang="c">
typedef struct {
typedef struct {
Line 129: Line 138:
     uint32 uid;
     uint32 uid;
     uint32 gid;
     uint32 gid;
     uint64 unk2[2];
     uint64 spare[2];
     uint32 blocks;
     uint32 blocks;
     int32 db[12];
     int32 db[12];
Line 136: Line 145:
</source>
</source>


=== Dirents ===
== Dirents ==
Each inode with a mode OR'd with 0x4000 points to block(s) of dirents. Dirents contain the name and type of files, directories, symlinks, etc. Each directory will have an associated dirent block containing at least the '.' and '..' special files, along with all other files and sub-directories in that directory. Dirents are 8-byte aligned. The <tt>entsize</tt> value will say the total length of this dirent. There is typically padding after <tt>name</tt> which can just be skipped.
 
Each inode with a the <tt>mode.dir</tt> bit set (<tt>mode |= 0x4000</tt>) points to block(s) of dirents. Dirents contain the name and type of files, directories, symlinks, etc. Each directory will have an associated dirent block containing at least the '.' and '..' special files, along with all other files and sub-directories in that directory. Dirents are 8-byte aligned. The <tt>entsize</tt> value will say the total length of this dirent. There is typically padding after <tt>name</tt> which can just be skipped.


If the type of the dirent is 3, it is a directory. Its <tt>ino</tt> value indicates the inode number (0 being the first inode). That inode will point to the dirent block for that directory so you can continue down the file system tree.
If the type of the dirent is 3, it is a directory. Its <tt>ino</tt> value indicates the inode number (0 being the first inode). That inode will point to the dirent block for that directory so you can continue down the file system tree.
Line 147: Line 157:
| 0x00 || ino || 0x4 || Inode index
| 0x00 || ino || 0x4 || Inode index
|-
|-
| 0x04 || type|| 0x4 || Type of entry. 2=file, 3=directory
| 0x04 || type|| 0x4 || Type of entry. 2=file, 3=directory, 4= . (link to current dir) , 5= .. (link to parent)
|-
|-
| 0x08 || namelen|| 0x4 || Length of filename (add 1 for 0-terminator)
| 0x08 || namelen|| 0x4 || Length of filename (add 1 for 0-terminator)
Line 157: Line 167:
| 0x11 + namelen || padding || variable || Padding so this structure is exactly <tt>entsize</tt> bytes.
| 0x11 + namelen || padding || variable || Padding so this structure is exactly <tt>entsize</tt> bytes.
|}
|}
<source lang="c">
<source lang="c">
typedef struct {
typedef struct {
Line 167: Line 178:
</source>
</source>


=== Finding the root ===
== Finding the root ==
 
The filesystem tree starts with the superroot, a sort of meta-directory that contains the root directory within it, along with something called a "flat_path_table". The superroot's inode is typically the first (zeroeth) inode entry in the table, but you should check <tt>PFS_HDR.superroot_ino</tt> for the actual index. The true root of the filesystem is the "uroot" directory within the super root.
The filesystem tree starts with the superroot, a sort of meta-directory that contains the root directory within it, along with something called a "flat_path_table". The superroot's inode is typically the first (zeroeth) inode entry in the table, but you should check <tt>PFS_HDR.superroot_ino</tt> for the actual index. The true root of the filesystem is the "uroot" directory within the super root.


=== flat_path_table ===
== flat_path_table ==
 
The flat_path_table is a file located above the root directory on every PFS image. It is simply a mapping of filename hashes to inode number, to increase the lookup speed for files:
The flat_path_table is a file located above the root directory on every PFS image. It is simply a mapping of filename hashes to inode number, to increase the lookup speed for files:
<source lang="c">
<source lang="c">
typedef struct {
typedef struct {
Line 180: Line 194:


The hashes are sorted in ascending order in the file. The hash function is given below:
The hashes are sorted in ascending order in the file. The hash function is given below:
<source lang="c">
<source lang="c">
uint32_t fptbl_hash(const char* filename){
uint32_t fptbl_hash(const char* filename){
Line 195: Line 210:
[https://www.google.com/patents/EP2878348A1?cl=en Patent explaining the flat path table]
[https://www.google.com/patents/EP2878348A1?cl=en Patent explaining the flat path table]


== Tools ==
= Tools =
* [https://github.com/maxton/GameArchives/releases/latest GameArchives/ArchiveExplorer] C# library/tool that supports opening and extracting from PFS images
* [https://github.com/maxton/GameArchives/releases/latest GameArchives/ArchiveExplorer by maxton] C# library/tool that supports opening and extracting from PFS images
* [https://github.com/maxton/MakePFS MakePFS by maxton] C# tool for creating PFS images

Latest revision as of 23:43, 27 December 2019

PFS (Pseudo/Playstation File System) is the file system used by (at least) downloadable content and games on the PS4. It is loosely based on the UFS (Unix File System) used in FreeBSD. PFS typically uses a 64kB block size, although the block size is configurable and specified in the file system header. The minimum block size is 4kB, and the maximum is 32MiB; the block size must be a power of 2.

Structure[edit | edit source]

There are four main sections in PFS:

  • Header (superblock)
  • Inode blocks
  • Directory blocks
  • Data blocks

Header/Superblock[edit | edit source]

Offset Value Size Notes
0x00 version 0x8 Always 1
0x08 format 0x8 Always 20130315
0x10 id 0x8
0x18 fmode 0x1
0x19 clean 0x1
0x1A ronly 0x1 If 1, this is a readonly filesystem
0x1B rsv 0x1
0x1C mode 0x2 Bit 0 = signed/unsigned, Bit 1 = 32/64 bit inodes
0x1E unknown 0x2
0x20 blocksz 0x4 The size of each block in the filesystem
0x24 nbackup 0x4 Seems to always be 0
0x28 nblock 0x8 Seems to always be 1
0x30 ndinode 0x8 Number of inodes in the inode blocks
0x38 ndblock 0x8 Number of data blocks
0x40 ndinodeblock 0x8 Number of inode blocks
0x48 superroot_ino 0x8
typedef struct {
    int64 version;
    int64 magic;
    int32 id[2];
    char fmode;
    char clean;
    char ronly;
    char rsv;
    int16 mode;
    int16 unk1;
    int32 blocksz;
    int32 nbackup;
    int64 nblock;
    int64 ndinode;
    int64 ndblock;
    int64 ndinodeblock;
    int64 superroot_ino;
} PFS_HDR;

Inodes[edit | edit source]

Inode table starts at the second block and continues for header.ndinodeblock blocks. There are only header.blocksz / sizeof(di_d32) inodes per block. (inodes will never cross a block boundary)

For an explanation of inodes, direct and indirect blocks, see inode pointer structure on Wikipedia. Most PFS images from PKGs have just one direct block pointer per file, and the file's data just takes up consecutive blocks. But if there is fragmentation on the file system, you will have to follow the direct and indirect block pointers.

There are actually 4 types of inodes that can be used in a PFS image, and the type is determined by the low two bits of themode field in the superblock. Most PFS images that you see, such as pfs_image.dat, will be using mode 0, which refers to non-signed 32-bit inodes.

Offset Value Size Notes
0x00 mode 0x2 Inode mode (bitwise OR of flags; file=0x8000, directory=0x4000), low 9 bits are file permissions
0x02 nlink 0x2 Number of links to this inode. For files, this is 1. For dirs, this is 1 plus the number of subdirectories (the .. entry links to the subdir's parent, increasing its nlink count).
0x04 flags 0x4 Bitfield of flags for compressed, readonly, etc.
0x08 size 0x8 Size in bytes of the entity
0x10 size_compressed 0x8 same as size for uncompressed PFS
0x18 times 0x30 Four 64-bit unix timestamps, followed by four 32-bit zeroes, which may be nanoseconds.
0x48 uid 0x4 User ID (zero for game PFS images at least)
0x4C gid 0x4 Group ID (zero for game PFS images at least)
0x50 spare 0x10 Probably the same as di_spare, always 0
0x60 blocks 0x4 Number of blocks occupied
0x64 db 0x30 Direct blocks
0x94 ib 0x14 Indirect blocks
typedef struct {
    // bitfields are from LSB to MSB
    struct {
        uint16 o_exec : 1;
        uint16 o_write : 1;
        uint16 o_read : 1;
        uint16 g_exec : 1;
        uint16 g_write : 1;
        uint16 g_read : 1;
        uint16 u_exec : 1;
        uint16 u_write : 1;
        uint16 u_read : 1;
        uint16 unk : 5;
        uint16 dir : 1;
        uint16 file : 1;
    } mode;
    uint16 nlink;
    struct {
        uint compressed : 1;
        uint unk : 3;
        uint readonly : 1;
        uint unk2 : 12;
        uint internal : 1;
    } flags;
    uint64 size;
    uint64 size_compressed;
    struct { 
        uint64 unix_time[4];
        uint32 time_nsec[4];
    } times;
    uint32 uid;
    uint32 gid;
    uint64 spare[2];
    uint32 blocks;
    int32 db[12];
    int32 ib[5];
} di_d32;

Dirents[edit | edit source]

Each inode with a the mode.dir bit set (mode |= 0x4000) points to block(s) of dirents. Dirents contain the name and type of files, directories, symlinks, etc. Each directory will have an associated dirent block containing at least the '.' and '..' special files, along with all other files and sub-directories in that directory. Dirents are 8-byte aligned. The entsize value will say the total length of this dirent. There is typically padding after name which can just be skipped.

If the type of the dirent is 3, it is a directory. Its ino value indicates the inode number (0 being the first inode). That inode will point to the dirent block for that directory so you can continue down the file system tree.

If the type of the dirent is 2, it is a file. Its inode will point you to the location(s) of the file data.

Offset Value Size Notes
0x00 ino 0x4 Inode index
0x04 type 0x4 Type of entry. 2=file, 3=directory, 4= . (link to current dir) , 5= .. (link to parent)
0x08 namelen 0x4 Length of filename (add 1 for 0-terminator)
0x0C entsize 0x4 Size of this whole struct, in bytes
0x10 name namelen + 1 Filename and 0-terminator
0x11 + namelen padding variable Padding so this structure is exactly entsize bytes.
typedef struct {
    int32 ino;
    int32 type;
    int32 namelen;
    int32 entsize;
    char name[namelen+1];
} dirent;

Finding the root[edit | edit source]

The filesystem tree starts with the superroot, a sort of meta-directory that contains the root directory within it, along with something called a "flat_path_table". The superroot's inode is typically the first (zeroeth) inode entry in the table, but you should check PFS_HDR.superroot_ino for the actual index. The true root of the filesystem is the "uroot" directory within the super root.

flat_path_table[edit | edit source]

The flat_path_table is a file located above the root directory on every PFS image. It is simply a mapping of filename hashes to inode number, to increase the lookup speed for files:

typedef struct {
    uint32 filename_hash;
    uint32 inode;
} path_table_entry;

The hashes are sorted in ascending order in the file. The hash function is given below:

uint32_t fptbl_hash(const char* filename){
  int i;
  uint32_t hash = 0;
  for(i = 0; filename[i]; i++) {
    // convert character to uppercase
    char c = (filename[i] >= 'a' && filename[i] <= 'z') ? (filename[i] - 32) : filename[i];
    hash = c + 31 * hash;
  }
  return hash;
}

Patent explaining the flat path table

Tools[edit | edit source]