ENCDEC Device Reverse Engineering

Introduction

• The following information was reverse engineered from LV1, Storage Manager in LPAR1, sb_iso_spu_module.self and sv_iso_spu_module.self.

Linux Driver ps3encdec

• I'm using this driver to set/clear my ATA and VFLASH keys.
• Tested on Linux 3.5.1.

Establish Secure Communication Channel

• First host and ENCDEC device exchange random numbers.
• From the exchanged random numbers host and ENCDEC device compute the session key.
• ENCDEC commands, e.g. to set ATA keys, are encrypted with the session key and AES-CBC-192.
• Before a secure communication channel is established, host and ENCDEC device use static AES-CBC-192 keys to encrypt communication data. The static keys can be found e.g. in sb_iso_spu_module.self or sv_iso_spu_module.self.
• Static ENCDEC keys depend on SB bus version. To get your SB bus version, read v2 of repository node SB bus id.
• During the communication, host and ENCDEC device use random IVs which are sent unencrypted together with encrypted payload.
• The ENCDEC commands, which are encrypted with the session key, contain magic 24 bytes which are checked by ENCDEC device and if some bits are not correct then the command is denied.
• The format of ENCDEC command to set ATA keys is slightly different from the ENCDEC command to set ENCDEC keys.