Retired > 2.0-RC Snapshot Feedback and Problems - RETIRED

HEADS UP: Textdumps coming in next snapshots to aid debugging

(1/3) > >>

jimp:
Starting with the next snapshots dated after this post, textdump support will be in all kernels for full installs.

Embedded systems lack the swap space necessary for dumps to work, so debugging with them will still require switching to a debug kernel.

So what are textdumps?
In short, they automatically capture information about a crash: The panic message, backtrace, and other info about the system state at the time of the crash.
For more in-depth detail, see here

After a crash, when your system reboots, you will be left with the textdump data in tar format under /var/crash, like so:

--- Code: ----rw-------  1 root  wheel  123904 Jan 27 17:01 textdump.tar.0
--- End code ---
If there are multiple textdumps, the number at the end will increment. The highest number (and of course latest date) are always the most recent data.

If asked by a developer, you can submit the whole tar file, or untar the file and copy/paste the individual files into forum attachments. To unpack the tar file on the router, you can do something like this:


--- Code: ---[2.0-BETA5][root@pfsense.localdomain]/root(1): mkdir crash
[2.0-BETA5][root@pfsense.localdomain]/root(2): cd crash/
[2.0-BETA5][root@pfsense.localdomain]/root/crash(3): tar xvf /var/crash/textdump.tar.0
x ddb.txt
x config.txt
x msgbuf.txt
x panic.txt
x version.txt
[2.0-BETA5][root@pfsense.localdomain]/root/crash(4): ls -l
total 122
-rw-------  1 root  wheel   5289 Jan 27 17:00 config.txt
-rw-------  1 root  wheel  49152 Jan 27 17:00 ddb.txt
-rw-------  1 root  wheel  65508 Jan 27 17:00 msgbuf.txt
-rw-------  1 root  wheel     16 Jan 27 17:00 panic.txt
-rw-------  1 root  wheel    138 Jan 27 17:00 version.txt

--- End code ---

Or copy the tar file off with scp and unpack it locally.

The files included in the dump are:

* config.txt - Kernel configuration
* ddb.txt - Captured DDB output (including the backtrace and other helpful info)
* msgbuf.txt - Kernel message buffer, should be roughly equivalent to the system.log file.
* panic.txt - Kernel panic message, if there was a panic
* version.txt - Kernel version string

You may still need to switch to a debug kernel to get more detail about locks.

Hopefully in the near future, none of this will be necessary :-)

David Szpunar:
Well that's nice to hear! I had a system I upgraded from a Nov. snapshot to one yesterday and it started locking up every 20 minutes, with no indication of what the problem was. Had to reinstall the Nov. snapshot and restore config, now it's fine again. I also had a VMware deployment I did last week that I had to revert back to the previous firewall because as soon as I put it in production and connected to cable modem, it started locking up. But no indication of what the problem was, so this is great to hear we'll be able to grab logs when that happens, thanks!

jimp:
Well, this won't help with hangs/lockups (unfortunately) but it will grab data from panics, which some people may just see as "spontaneous reboots" because they may not see the panic message on the console due to the automatic reboot.

At some point before release these dumps will probably be scaled back to only being in the dev kernel again as the changes required to make this work increased the kernel sizes significantly.

David Szpunar:
Boo...any suggestions on the hangs/lockups, or am I out of luck? I keep meaning to post a question in the forum but a) I have no actual information, the System Logs are clean and b) I haven't had time. I assume I'd need a debug kernel at the least but I haven't messed with those at all, and I don't know if they'd help.

jimp:
Join the party on this thread:

PFSense 2.0 Beta5 1/19 build system locks up

It's something in the FTP proxy, it's being actively pursued and should hopefully be fixed soon...

Navigation

[0] Message Index

[#] Next page

Go to full version