Editing Validating flash dumps

[[Category:Hardware]][[Category:Software]]<div style="float:right">[[File:Dumpchecker.png|200px|thumb|left|Dumpchecker]]</div>

= Why so important? =
Whenever you want to mess with flash (NAND or NOR), you have to '''make sure you are getting 100% correct, valid, verified dumps.'''

{{warning|content=If you do not validate your flash, and for some reason(s) the dump is invalid/bad, and you are rewriting to flash, you have <span style="background-color:white; color:red;">no way of restoring</span> the console hence making a perma-brick (unrecoverable brick).
}}
So what can we use to prevent that, and make sure our dumps are as proper as we can check?

= Visual inspection =
<div style="float:right">[[File:HxD.png|thumb|left|HxD - hexeditor]]</div>

Open up the flashdump (make it unified first if it is still 2 seperate NAND dumps) with a hexeditor (e.g. [http://mh-nexus.de/en/hxd/ HxD])<br />and compare what you see with the [[Flash]] page.

== Statistical analysis ==


One neat option in HxD is the analysis option. Open dump file and select Statistics in the Analysis menu. Mouse over the blue graph bars of [[:File:HxD-Analysis-Statistics-mouseover-00.jpg|00]] and [[:File:HxD-Analysis-Statistics-mouseover-FF.jpg|FF]] <br />

{{Dumpstatistics}}

If you do not have HxD many other hex editors have a statistics option. If you want a standalone application that specifically informs you on the validity of the given statistical range (NOR only) use the [http://betterwayelectronics.com/files/BwE%20NOR%20Statistics%20v1.4.exe BwE NOR Statistics application].

== Bytereversed ==
   00 00 00 00 AC 0F FF E0 00 00 00 00 AD DE EF BE  (HEX)   and   saceru_eoldare  (Text)   RAW/PS3 readable - this is E3 default byteorder
   instead of
   00 00 00 00 0F AC E0 FF 00 00 00 00 DE AD BE EF  (HEX)   and   asecure_loader  (Text)   REV/Human readable - this is Progskeet default byteorder
You can use either the bytereverse option in the the flasher application and redump, or bytereverse the current dump with [[Validating_flash_dumps#Flowrebuilder|Flowrebuilder]] or [http://www.neillcorlett.com/cmdpack/ cmdline tools] (using record to swap: 2).

== Only zeroes ==
 00 00 00 00 00 00 00 00 00 where data should've been
* Seems reading is a problem, mostly caused by bad wiring/clip seating -> check

== patterned non zeroes / patterned non ff's ==
   30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
   00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80   where zeroes should've been

   EF FF EF FF EF FF EF FF EF FF EF FF EF FF EF FF   where FF's should've been

* Seems reading is a problem, mostly caused by bad wiring/clip seating -> check
* Can also be caused by bad grounding, too low Vcc, interference or too long wires -> check

== patterned FF's ==
   FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF   repetive banding of FF's where data or zeroes should have been
* Seems reading is a problem, mostly caused by bad wiring/clip seating -> check

== malformed headers / filenames / regionnames ==
   IJI              instead of   IFI
   asecure_loaher                asecure_loader
   eIIH                          eEID
   cESH                          cISD
   cCSH                          cCSD
   trvg_prk0                     trvk_prg0
   trvg_prk1                     trvk_prg1
   trvg_pkk0                     trvk_pkg0
   trvg_pkk1                     trvk_pkg1
   cztrm                         cvtrm
   mitldr                        metldr
* Seems reading is a problem, mostly caused by bad wiring/clip seating -> check

== absent files / regions ==
   00FC0000  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF  ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ    missing bootldr in NOR

   00000000  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF  ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ    missing bootldr in NAND
* Seems reading is a problem, mostly caused by bad wiring/clip seating -> check

== malformed files / regions ==
   00FC0000  28 B4 4F D2 F9 3F BC 43 28 B4 4F D2 F9 3F BC 43  (´OÒù?¼C(´OÒù?¼C    corrupted bootldr in NOR (it should always start with 00 00 there)
* Seems reading is a problem, mostly caused by bad wiring/clip seating -> check

== Repetitions ==
take 16 bytes of an offset that should be unique, e.g. the 0x10 offset which normaly has the DEADBEEF magic header :
   NOR: 0x0000010 00 00 00 00 0F AC E0 FF 00 00 00 00 DE AD BE EF  //  NAND: 0x040010 00 00 00 00 0F AC E0 FF 00 00 00 00 DE AD BE EF
And find all instances in hexeditor <br />
When it says found in 0x10 and 0x10010 you substract them (0x10010 - 0x10 = 0x10000) <br />
So that means there is an error with setting 0x10000 (only one bit set!) low <br />
because the programmer applies 0x10010 but the actual data on the address lines (received by the nor flash) is 0x10 <br />

Other semistatics: http://pastebin.com/7QMpLSBw (not all unique, but can be used to check offsetting by bad addresslines)

=== Checking again with other unique 16 byte sequence ===
   617365637572655F6C6F616465720000 (asecure_loader in hex)
Same check, found at 0x420 and 0x10420 (0x10420-0x420 = 0x10000)

=== OK, you've found the pattern, now how do we deduct from that the faulty line? ===
address is 2 bytes. 0x10 in dump with hexeditor is in reality 0x8 on the address bus because we read 2 bytes at once <br />
so the address is no longer 0x10000 but 0x8000 <br />
0x8000 (hexadecimal = base 16) = 1000 0000 0000 0000 (binary = base 2) <br />
which means the 16th bit = 1, we start counting at A0, so the 16th is A15 <br />

conclusion : A15 is bad


<!--// Well here is a program that will verify the size of the dump, check the header 0xDEADBEEF and its repetation and shows you which address line is bad and it will also check if the console is downgradable or not by looking at the bootloader data. So with this tool you have all in one checker but even we will say you to verify the dump manually or using flowrebuilder also, Link to the program: http://www.sendspace.com/file/byubqj

This program will check for bad file size as well as if the magic header are repeated or not and also calculate the difference of repeated header and tell you which address line is bad. If the dump is good it will tell you weather the console is downgradable or not? //-->

==== Quick lookup table ====
{| class="wikitable"
|-
! repetition !! addressline 'stuck'
|-
| 0x2 || A0
|-
| 0x4 || A1
|-
| 0x8 || A2
|-
| 0x10 || A3
|-
| 0x20 || A4
|-
| 0x40 || A5
|-
| 0x80 || A6
|-
| 0x100 || A7
|-
| 0x200 || A8
|-
| 0x400 || A9
|-
| 0x800 || A10
|-
| 0x1000 || A11
|-
| 0x2000 || A12
|-
| 0x4000 || A13
|-
| 0x8000 || A14
|-
| 0x10000 || A15
|-
| 0x20000 || A16
|-
| 0x40000 || A17
|-
| 0x80000 || A18
|-
| 0x100000 || A19
|-
| 0x200000 || A20
|-
| 0x400000 || A21
|-
| 0x800000 || A22
|-
| 0x1000000 || A23
|-
| 0x2000000 || A24
|-
| 0x4000000 || A25
|-
| 0x8000000 || A26
|-
| 0x10000000 || A27
|-
| 0x20000000 || A28
|-
| 0x40000000 || A29
|-
| 0x80000000 || A30
|-
|}

== Check metldr+bootldr sizes ==

{{warning|content=If they are even missing or bad, its a sure road to permabrick
}}
<br />
{{metbootldr}}

= Flowrebuilder =
<div style="float:right">[[File:FlowRebuilder.png|thumb|left|Flow Rebuilder]]</div>

== Links to versions ==
http://www.ps3devwiki.com/files/flash/Tools/Flowrebuilder/

 4.2.3.0 changelog
 -----------------
 added support to manage NAND preloader dumps
 message user about the type of dump
 message the user if bootloader are missing
 auto-recognize if dump is normal or byte swapped and automanage them

* [http://www.ps3devwiki.com/files/flash/Tools/Flowrebuilder/FlowRebuilder%20v.4.2.3.0.exe FlowRebuilder v.4.2.3.0.exe (367 KB)] // [http://www.mirrorcreator.com/files/1HWLWYIY/FlowRebuilder_v.4.2.3.0.exe_links mirror] [http://www.mirrorcreator.com/files/1NYWARZT/FlowRebuilder_v.4.2.3.0.exe_links mirror2]
* [http://www.ps3devwiki.com/files/flash/Tools/Flowrebuilder/FlowRebuilder%20v.4.2.2.0.exe FlowRebuilder v.4.2.2.0.exe (389 KB)] // [http://www.mirrorcreator.com/files/7G5JLEK9/FlowRebuilder_v.4.2.2.0.exe_links mirror] [http://www.multiupload.nl/0ND4D0QW9Q mirror 2] [http://www.mirrorcreator.com/files/TJKTQQWB/FlowRebuilder_v.4.2.2.0.exe_links mirrors]
* [http://www.ps3devwiki.com/files/flash/Tools/Flowrebuilder/FlowRebuilder%20v.4.2.1.2.exe FlowRebuilder v.4.2.1.2.exe (313.50 KB)] // [http://www.mirrorcreator.com/files/08SGK09K/FlowRebuilder_v.4.2.1.2.exe_links mirror]
* [http://www.ps3devwiki.com/files/flash/Tools/Flowrebuilder/FlowRebuilder%20v.4.2.1.1.exe FlowRebuilder v.4.2.1.1.exe (473.57 KB)] // [http://www.mirrorcreator.com/files/GUUUKNWE/FlowRebuilder_v.4.2.1.1.exe_links mirror]
* [http://www.ps3devwiki.com/files/flash/Tools/Flowrebuilder/FlowRebuilder%20v.4.2.0.1.exe FlowRebuilder v.4.2.0.1.exe (474.01 KB)] // [http://www.mirrorcreator.com/files/C4U1C3OO/FlowRebuilder_v.4.2.0.1.exe_links mirror]
* [http://www.ps3devwiki.com/files/flash/Tools/Flowrebuilder/FlowRebuilder+v.4.1.3.2.exe FlowRebuilder v.4.1.3.2.exe (459.95 KB)] // [http://www.mirrorcreator.com/files/1UEPOIHX/FlowRebuilder_v.4.1.3.2.exe_links mirror]
* [http://www.ps3devwiki.com/files/flash/Tools/Flowrebuilder/FlowRebuilder%20v.4.1.0.0.exe FlowRebuilder v.4.1.0.0.exe (454.85 KB)] // [http://www.mirrorcreator.com/files/EC2ZFONW/FlowRebuilder_v.4.1.0.0.exe_links mirror] (don't use, for historic reasons listed)

==Flowrebuilder options==
* (NAND only) UNSCRAMBLE then interleave two NAND flashes into one unified dump - (4.1.0.0 and higher) : Makes a unified dump.bin from 2 seperate NAND flash dumps.
** In the second step it also extract the content of the unified dump. Make sure it extracts correctly (it will give no warning if it fails!) and all the needed files are there.
* (NAND only) RE-SCRAMBLE a modified dump then de-interleave it into two new flashes - (4.1.0.0 and higher) : Splits the unified dump.bin into 2 new seperate NAND flash dumps.
* Byte reverse and EXTRACT a NOR  dump file - (4.1.0.0 and higher) : First byte reverse the single dump.bin then extract NOR content.
* EXTRACT a Byte reversed NOR dump or an interleaved and unscrambled NAND dump - (4.1.0.0 and higher) : Extract the single dump.bin.
* Byte reverse and PATCH a NOR dump - (4.2.1.1 and higher).
* PATCH a Byte reversed NOR dump or an interleaved and unscrambled NAND dump - (4.2.1.1 and higher).
* Byte reverse a dump - (4.2.1.1 and higher).

Note:
* <span style="text-decoration: line-through;">when getting [http://pastie.org/3656024 errors with autopatcher], use Winskeet to patch the <abbr title="validated interleaved and unscrambled NAND dump">unified dump</abbr> and then Flowrebuilder to Rescramble it.</span> -> Fixed in FlowRebuilder v.4.2.1.2

* Flowrebuilder 4.2.2.0 error in Wine : http://pastie.org/private/nxf7tutolf9jjojskxw

== Extracted flash content files ==
<div style="float:right">[[File:Nand-extract-noerror.png|200px|thumb|left|Flowrebuilder extraction no error]]<br />[[File:Nand-extract-error2.png|200px|thumb|left|Flowrebuilder extraction error]]</div>

<span style="background-color:white; color:red;">(make sure they are all there, flowrebuilder will not give warning when it fails!)</span>:
* bootloader_0
* bootloader_1 (NAND only)
* cCSD
* cISD
* creserved_0 (NAND only)
* cvtrm
* eEID
* trvk_pkg
* trvk_prg
* \asecure_loader\metldr
* \ros\<nowiki>[two seperate folders named to FW version]</nowiki>\[[CoreOS|CoreOS files]] (19 up to 25 files, depending the FW version)

'''Notes:'''
*  if it only extracted bootloader_0 + bootloader_1, check that both NANDs are dumped correct (known error with flashers that has bug with second NAND channel to read), and that high/low where selected correct.

* Do not use spaces in the folderpath, or Flowrebuilder will not extract the files.<!--// http://www.mediafire.com/?229seszf07votsv domelec //-->

* <span style="text-decoration: line-through;">Bug workaround: delete previous extraction attempts to prevent Flowrebuilder to always error. -> [http://pastie.org/3744043 bugreport]</span> -> Fixed in FlowRebuilder v.4.2.2.0

== From unified to seperate low/high NAND files ==
If for some reason you lose the original seperate low/high NAND files and only have the <abbr title="unscrambled then interleaved 2 NAND to unified dump">unified</abbr>, you need donor files to regenerate them. 
* [http://www.ps3devwiki.com/files/flash/dumps/NAND0-1.rar NAND0-1.rar] [http://www.mirrorcreator.com/files/LTVIJEAK/NAND0-1.rar_links mirror]
extract and in Flowrebuilder select the option "RE-SCRAMBLE a modified dump then de-interleave it into two new flashes"
* Select the NAND backup "Flash 0" (TOP) file : here drop the 1bkp.bin
* Select the NAND backup "Flash 1" (BOTTOM) file : here drop the 2bkp.bin
* Select the Input NAND interleaved Modified file : here drop your unified flash .bin file
and click Execute Operation

This will result in making 2 new deinterleaved files you could use for reflashing (it is recommended to alway use '''''your own''''' deinterleaved sourcedumps, this method is only mentioned as last resort for people that lost their backups).

= BwE NOR Validator =

This all in one validator/patcher will interpret the byte orientation, patch for 3.55/3.56/4.40 and will then output the results of 2400+ validations via HTML. <br><br>
[http://betterwayelectronics.com/Latest.html Output Example] 

=== How? ===
1 - Place your .bin/s in the same folder as the validator.<br>
2 - Run the validator and press Start.<br>
3 - A console will appear asking you to select your dump (if you have more than one in the folder).<br>
4 - Make your selection and or select if you watch to patch it (either in its current byte orientation or the opposite).<br>
5 - Wait patiently.<br>
6 - Press Enter at the end to launch the output.<br>

=== Areas Of Validation ===
 * Statistics
 * First Region Header
 * Flash Format
 * Flash Region
 * Asecure_Loader/Metldr
 * Asecure_Loader/Metldr Corrupt Sequences
 * Asecure_Loader/Metldr Encrypted Statistics
 * EID
 * EID0
 * EID2
 * EID3
 * EID5
 * IDPS
 * CISD
 * CISD0
 * CISD1
 * CISD2
 * CCSD
 * CCSD0
 * TRVK_PRG0
 * TRVK_PRG1
 * TRVK_PKG0
 * TRVK_PKG1
 * ROS0
 * ROS1
 * ROS0/1 AuthID's/MD5's
 * Revoke/CoreOS MD5's
 * CVTRM/VTRM0
 * VTRM 1
 * Second Region Header
 * Second Region Block 0
 * Second Region Block 1
 * CELL_EXTNOR_AREA
 * Lv0ldr/Bootldr
 * Lv0ldr/Bootldr Corrupt Sequences
 * Lv0ldr/Bootldr Statistics
 * Minimum Version
 * File Digest Keys
 * PerConsole Nonce
 * Corrupt Sequences
 * Repetition
 * Authenticiation IDs

=== Download ===
[http://betterwayelectronics.com/files/BwE_NOR_Validator.rar BwE NOR Validator v1.22] 

Application MD5: 5802A5D46BD7D694F2F11E67B038F9D4<br>

==== Changelog ====
* 1.22 - 16/04/2013 : added 115 more validations + changed statistic range for bootldr + other small boring changes
* 1.21 - 11/04/2013 : patch3 error fix.
* 1.20 - 10/04/2013 : added new console data, removed 3.56 patch (replaced with 4.40), added protection against using old validator, changed corruption check (again!), added quick info for console.
* 1.19 - 09/04/2013 : changed metldr statistic range, minver check (to suit refurbished ps3s), corruption changes
* 1.18 - 08/04/2013 : upgraded cisd/cell_ext_nor_area/metldr validations to suit unique metldr.2 revision, changed corruption output (again).
* 1.17 - 08/04/2013 : changed repetition check, changed corruption output, bugfix
* 1.16 - 02/04/2013 : added rogero's 4.40 patch, changed options, added 25+ md5's, changed stats range and id check
* 1.15 - 25/03/2013 : added 4.40 ofw information and optimised some code
* 1.14 - 19/03/2013 : improved validation of the flash-region table
* 1.13 - 18/03/2013 : better handling for metldr.2, more id detections and md5s, added byte reversal option for experimenting with E3.
* 1.12 - 16/03/2013 : md5 bug fix, changed id detections and general improvements
* 1.11 - 08/03/2013 : improved patching structure, added 3 musketeers patch (3.56 patching), code optimisation
* 1.10 - 04/03/2013 : added 25+ validations + changed results & outputs + bugfix
* 1.09 - 02/03/2013 : improved corruption checks for metldr/bootldr + more validations + old coreos bug fix (again) + changed some results
* 1.08 - 29/02/2013 : added timeout for version check + added 16bit corrupt sequence check + fixed long outputs + fixed metldr ident bug + changed some warning/danger results + changed 00/ff results
* 1.06 - 27/02/2013 : fixed bug when handling old coreos versions + fixed .self md5 list
* 1.05 - 25/02/2013 : added approx 220 more validations + changed statistic ranges + latest version check + fixes to metldr/bootldr
* 1.02 - 22/02/2013 : more md5's & authid checks + changed some results.
* 1.01 - 16/02/2013 : authid check bugfix
* 1.00 - 15/02/2013 : first public release

=== Notes/Warning ===
Using this program is [[NOT]] a 100% way of validating your dumps (because ''nothing'' is), but it is a good way of quickly seeing if it is worth checking further. This program will NOT work with NAND and will not check anything other than what is stated. ALWAYS CHECK MANUALLY!

Use at own risk! Valid dumps may be invalid - Invalid dumps may be valid. <br>
There are almost infinite variations of each dump! Have fun and good luck! <br>

Any updates will be posted here first. <br>

Report any bugs or issues to bwe@betterwayelectronics.com or directly to BwE @ irc.efnet.org #ps3downgrade <br>

Give credit if you are using this for other people!<br>

Made in Australia!<br>

=== Gallery ===
<gallery>
File:Sshot-14.jpg
File:Sshot-12.jpg
File:Sshot-4.jpg
File:Screenshot.jpg
File:Sshot-147.jpg
</gallery>

= Nor Inspector =
Another Tool for validating your NOR Dump is "Nor Inspector".

It will display most common information about your dump and the console. <br>

=== How-to use ===
1) start<br>
2) drag'n'drop your dump into the app<br>
3) on errors check out status-tab<br>

=== Features ===
Other Features:

 * Drag'n'drop [[CoreOS]] / [[Revokation]] Package to replace
 * Hex-Region Viewer
 * Byte reverse dumps supported
 * Error Logging in Status-Tab (OK/ERR) 
   "The Tab is green, the dump is clean!"
   ...

=== Download ===
[https://dl.dropbox.com/s/aeqvmm7e4boe51c/NorInspector001.rar Nor Inspector 0.0.1] (Requires .NET Framework 2.0)

=== Notes/Warning ===
Using this program is [[NOT]] a 100% way of validating your dumps (because ''nothing'' is), but it is a good way of quickly seeing if it is worth checking further. This program will NOT work with NAND and will not check anything other than what is stated. ALWAYS CHECK MANUALLY!


=== Gallery ===
<gallery>
File:NorInspector-tab-General Information.png|NorInspector - tab : General Information
File:NorInspector-tab-More Information.png|NorInspector - tab : More Information
File:NorInspector-tab-Region Viewer.png|NorInspector - tab : Region Viewer
File:NorInspector-tab-Status.png|NorInspector - tab : Status
</gallery>

= Known bad guides (with obvious faults in them that lead to permanent bricks) =
== Baileys (&gt;10 bricks) ==
* <span style="text-decoration: line-through;">http://psx-scene.com/forums/f263/ultimate-fool-proof-guide-ps3-hardware-downgrading-104398</span>
* <span style="text-decoration: line-through;">http://www.ps3hax.net/showthread.php?t=39766</span>
'''Do not use''', not only does it deviate alot (16x) from wiki safeguarded method, it skips important steps that prevent users from bricking their console.

== E3 FLASHER INSTALLATION and DOWNGRADE TUTORIAL (Khalkedon) ==
* <span style="text-decoration: line-through;">http://www.youtube.com/watch?v=WHNw2E4tU6c&t</span> '''Do not use'''
=== What is wrong / could be better ? ===

# cutting knife VS plastic to remove the bolt protectors
# cellophane 3M tape in hot environment
# ribbon flat cable not at 90 degrees position
# no fixating of clip (coin/foam filler to keep the clip more pressed)
# no tristate wire soldered
# not checking clip connection, before closing console
# no repasting of RSX and CellBE
# not using torc moment screwdriver, but electric drill
# uneven pressure when mounting frontboard
# greasy fingerprints all over the motherboard (bonus fingerprint: under cellophane tape)
# dual sata adaptor is used (not recommended, as ps3 host does not support SATA Port Multiplier)
# E3 is ´wobbly´ because no cardboard filler was used to prevent movement (bracket design error)
# not using the 'Samsung' multiconsole update for E3 and no proper SHA1 identification of the correctness of file
# using 001000 for dumping, not using flashfun, thus risking busy flashbus on a multithreaded system
# not validate content of dumps, only size
# doing a restore (will permabrick console if clipped bad)
# when unpluging the power cord at 19:09, insuficient time to dischargue capacitors of power supply (a real reset needs more time)
# using e3_downgrader.bin (nicknamed e3_bricker for obvious reasons) instead of prepatching
# misnaming ¨2nd usb port¨ -> it's usb00000   because we count from right to left in ps3 (rightmost, closest to BD)
# no proper identification of lv2diag.self (there exist many!) and PS3UPDAT.PUP to be used (SHA1)
# after FSM reinstall, no boot to XMB to check it booting proper ''with FSM active''
# not setting proper time, corrupting node database unneeded
# fix permission errors (archaic, not needed)
biggest error : not reading the (E3) downgrade guide on wiki

challenge: ''can you name another error?''

== Progskeets official downgrade guides ==
* http://www.progskeet.com/downloads/NOR_eng.pdf
** Only checks statistics, no validation at all
* http://www.progskeet.com/downloads/NAND_eng.pdf
** Checks statistics, Extracts dump but does not check its content, no validation.
** The use of Flowrebuilder, to make a unified dump from 2 NAND dumps is a bit strangely placed in the guide

= Word of Warning =
When using validators and bricking your console as result, there are only 2 people to blame :
* the one making the validator (for not properly validating, warning users and telling ''what'' is wrong and/or ''how'' it could be solved);
* the enduser for flying blind and getting lazy/dumb with it.