Extracting the recovery image from my Router/ONT

Fri, Jan 19, 2018

UART on the board, complete with headers

Finding the UART on a Broadcom device usually enables access to its CFE, and consequently access to the device’s flash memory

In the past, device firmware files were typically made available to users by their respective manufacturers. A recent trend has emerged where ISPs are completely managing their consumer premise equipments, including provisioning and firmware updates. As such, publicly available firmware files may be scarce or unavailable for some new ONTs or routers.

It may be possible to extract the current firmware (or recovery image, stored in alternate bank) in the right situation:

Device is based on a Broadcom processor
UART pins/pads/headers are available
CFE (Broadcom’s bootloader) is recent enough
CFE is not deliberately crippled by manufacturer
Prerequisite hardware is available to access the UART

Identifying the UART

Identifying the UART was a straightforward process for my board, as it already had standard 0.1" headers along with proper silkscreen labels (GND, RX, TX).

For difficult devices, devttys0 has an excellent post on locating and identifying UART pins/pads on a board.

The UART settings are typically 8N1 at 115200 baud (occasionally 9600 baud for some devices).

Booting with the UART connected

There are some messages to indicate the device’s state as it boots:

CFE bootloader message should appear

CFE version 1.0.38-118.5 for BCM96838 (32bit,SP,BE)

CFE will wait for timeout

*** Press any key to stop auto run (1 seconds) ***

Kernel is decompressed and started

Decompression LZMA Image OK!
Entry at 0x8043d6e0
Starting program at 0x8043d6e0
Linux version 3.4.11-rt19 (buildczar@lgbuild3.largo.zhone.com)

Understanding the memory layout

When the kernel initializes the NAND during a normal boot, it may display a message describing the memory layout

Creating 8 MTD partitions on "brcmnand.0":
0x000003d80000-0x000007ae0000 : "rootfs"
0x000000020000-0x000003d80000 : "rootfs_update"
0x000007b00000-0x000007f00000 : "data"
0x000000000000-0x000000020000 : "nvram"
0x000003d80000-0x000007ae0000 : "image"
0x000000020000-0x000003d80000 : "image_update"
0x000000000000-0x000008000000 : "dummy1"
0x000000000000-0x000008000000 : "dummy2"

When the above data is visualized, the output would be similar to this:

Memory layout

It becomes clear that the memory is predominantly occupied by the identically-sized rootfs/image (gray) and its recovery image (orange). To obtain the recovery image, data from 0x0020000 to 0x3d80000 has to be dumped from the NAND.

Working with CFE

When booting with the UART connected, there will be a moment where the CFE waits for input to interrupt the boot process.

*** Press any key to stop auto run (1 seconds) ***

Mash your keyboard until a CFE prompt appears

CFE>

Help is available via the help command

CFE> help
Available commands:

m                   Load manufacturing factory defaults
v                   Display cfe & application firmware version(s).
x                   Change extra partitions size
find                Find string in NAND
comp                Compare NAND blocks
fb                  Find NAND bad blocks
dn                  Dump NAND contents along with spare area
phy                 Set memory or registers.
sm                  Set memory or registers.
dm                  Dump memory or registers.
db                  Dump bytes.
dh                  Dump half-words.
dw                  Dump words.
w                   Write the whole image start from beginning of the flash
e                   Erase NAND flash
ws                  Write whole image (priviously loaded by kermit or JTAG) to flash .
r                   Run program from flash image or from host depend on [f/h] flag
p                   Print boot line and board parameter info
c                   Change booline parameters
i                   Erase persistent storage data
a                   Change board AFE ID
b                   Change board parameters
reset               Reset the board
pmdio               Pseudo MDIO access for external switches.
spi                 Legacy SPI access of external switch.
force               override chipid check for images.
help                Obtain help for CFE commands

The dn command is useful for dumping data from the NAND

CFE> help dn

  SUMMARY

     Dump NAND contents along with spare area

  USAGE

     eg. dn [block] [page] [total pages to view] or dn [address] [total pages to view]

*** command status = 0

The output of dn appears in a hexdump-like format:

CFE> dn 0x7b00000 2050
------------------ block: 984, page: 0 ------------------
07b00000: 1985e002 00000af8 9cd73513 00000004    ..........5.....

07b00010: 00000282 00008180 00000000 00006000    ..............`.

07b00020: 59efb809 59efb809 59efb809 0000554c    Y...Y...Y.....UL

07b00030: 00000ab4 00000ab4 00000000 981971e3    ..............q.

Dumping the recovery image from the NAND

Recovery image in binwalk

The process involves a few steps:

Running dn with the desired address and a sufficiently large number of pages
Saving the UART output. (I used PuTTY’s logging feature)
Parsing the hexdump format to emit a binary file

As the data is transferred over a measly 115.2kb/s, the dump will likely take hours to complete.

After it is dumped and converted into a binary file, binwalk should (hopefully) recognize it as a filesystem such as squashfs/ubifs/jffs2. A really neat feature of binwalk is the -e command, where binwalk -e on your binary will automatically extract the filesystem contents. Take note that some binwalk dependencies (such as jefferson) do not work well with Windows.

At this point, you should have access to the files in your ONT’s recovery image. Congratulations! ^_^

Why though?

Access to the firmware enables static binary analysis, to verify if earlier security issues were properly fixed - previously, shell commands were executed without prior sanitization.

I found some issues with the earlier firmwares of my ONT’s device family:

With a known network address, it was possible to get (unauthenticated) remote code execution as root
All ONTs also had an network interface connected to the ISP’s internal management network, and were visible to each other

Customer’s provisioning data (legal name, full address, SSID, keys) were also stored in the device’s nvram, further excerbating the issue.

Unfortunately, the issues were never completely fixed; I formally reported the issue to my ISP in July ‘14, and my (still-vulnerable) ONT was replaced in Q4 ‘17. The vendor/ISP’s approach to this issue was somewhat concerning, hence I believed it was wise to personally verify if the issues were resolved (via static analysis).

Thoughts

Dumping firmware over 115200 baud is ridiculously slow. If there was a means to use the network interface instead (tftp in CFE?), I would love to hear about it.

The sanitization routine blocks these characters:

|;`&()><

Interestingly, the $ character is still unfiltered. Can $ be used to bypass this filter?

jg