next »

[eweri]

Hello!

Just installed pfSense-2.1-DEVELOPMENT-4g-i386-nanobsd-20120425-2326.img fresh (no update) on my 8GB Kingston CF with my ALIX board and it is still very slow.
Well it is more like it is sleeping for minutes than does its job until it is sleeping for minutes again.

For example: I am connected to the serial console and the first sleep happens when it tries to configure the WAN interface - boot process hangs for at least 2 minutes. After pfsense is completely up and running I select "Assign interfaces" from the menu and try to assign the interfaces in the right order. Well when I tried to detect interfaces automatically it takes very long, until I get any response.
After every interface is assigned it takes very long "writing configuration" but after that my WAN got no IP-address (should use DHCP).
Than I selected "Reboot system" and once again it takes ages ;-) until the system reboots.

But one thing is better than the snapshots before - now after reboot my WAN-interface got an IP-address from my DHCP server.

Hope this helps to solve this problem, bye,
eweri

[moullas]

re-flashed today to latest snap... still extremely slow, when re-mounting as ro

Btw, my CF is a Lexar 4GB if it matters.. are there any logs that may shed some light?

[jimp]

To add a couple more data points:
* Enabling TRIM on the slice - no effect, still slow
* Enabling softupdates on the slice - no effect, still slow
* Running sync;sync;sync; before mounting ro - no effect, still slow
* The mount process sticks in biowr state, which is disk i/o
* While mount is stuck, the system is somewhat unresponsive - can't run a process or connect with ssh, but does appear to continue routing.

[jimp]

I went ahead and opened a redmine ticket for this:
https://redmine.pfsense.org/issues/2401

[markky]

...

Many have also suggested that there's a difference in behaviour between an upgrade an a fresh install of the flash card.  I doubt it actually has any impact, but I'll run that test tomorrow on the cheapo card to eliminate/confirm it.

 - Mark

As expected, the slow CF card still exhibits stalls in mount rw -> ro irrespective of whether the image is an upgrade or a complete reimage of the flash.

As far as I can remember, my slow CF card can be written at just under 4 MB/s from the dev virtual machine.  The sandisk card by comparison writes at over 5MB/s.

 - Mark

Within half an hour of putting the 4801 in production with a sandisk 8G card instead of the previous slower 4G card, the system started exhibiting "stall" conditions.

Unfortunately, my production placement makes serial access a pain (and hence kernel debugger) so I haven't garnered much since.

I'm at the point where I'm thinking I'll just rebuild the kernel and use unionfs to reduce flash wear and tear and do the occasional sync to either underlaying fs or a persistent overlay.

 - Mark

[markky]

I went ahead and opened a redmine ticket for this:
https://redmine.pfsense.org/issues/2401

Here's a bit more detail for what you described.  The fact that flushfiles is flushing out so much on a quiescent filesystem is rather suspect (as a counter point, run lsof and look for writeable files).


The next line is a tty interrogation with ^T
load: 0.22  cmd: mount 2123 [biowr] 5.82r 0.02u 0.08s 7% 1044k

This is the alternate ddb entry, "~^b"

~KDB: enter: Break sequence on console
[thread pid 10 tid 64003 ]
Stopped at      kdb_enter+0x3a: movl    $0,kdb_why

# Just what is the hung mount doing...
db> bt 2123
Tracing pid 2123 tid 64074 td 0xc25612e0
sched_switch(c25612e0,0,104,191,10e78686,...) at sched_switch+0x399
mi_switch(104,0,c0c8be83,1eb,4c,...) at mi_switch+0x1da
sleepq_switch(c25612e0,0,c0c8be83,260,0,...) at sleepq_switch+0x15f
sleepq_wait(cf240fd0,4c,c0c92fe4,0,0,...) at sleepq_wait+0x63
_sleep(cf240fd0,c212c524,4c,c0c92fe4,0,...) at _sleep+0x372
bwait(cf240fd0,4c,c0c92fe4,cf240fd0,d412a69c,...) at bwait+0x6f
bufwait(cf240fd0,cf240fd0,df,cf240fd0,c22fb5e4,...) at bufwait+0x48
bufwrite(cf240fd0,0,c0cb0aac,772,0,...) at bufwrite+0x165
ffs_bufwrite(cf240fd0,c22fd600,80,1000,0,...) at ffs_bufwrite+0x27b
ffs_update(c238610c,1,c0cb10b8,162,d412a798,...) at ffs_update+0x2ac
ffs_syncvnode(c238610c,1,c25612e0,c2320110,c23861b4,...) at ffs_syncvnode+0x48f
ffs_fsync(d412a858,c238610c,0,c238610c,d412a880,...) at ffs_fsync+0x27
VOP_FSYNC_APV(c11dcc20,d412a858,c0c955dc,9cb,0,...) at VOP_FSYNC_APV+0xa5
vflush(c22f2c94,0,6,c25612e0,c22e0200,...) at vflush+0x2b4
ffs_flushfiles(c22f2c94,6,c25612e0,e6,c095ca37,...) at ffs_flushfiles+0x7b
ffs_mount(c22f2c94,0,c0c94deb,3ed,0,...) at ffs_mount+0x452
vfs_donmount(c25612e0,10090003,c286aa00,c286aa00,bfbfdca9,...) at vfs_donmount+0x1012
nmount(c25612e0,d412acec,c25612e0,c1214c80,207,...) at nmount+0x64
syscall(d412ad28) at syscall+0x24e
--More--^M        ^MXint0x80_syscall() at Xint0x80_syscall+0x21
--- syscall (378, FreeBSD ELF32, nmount), eip = 0x280e746b, esp = 0xbfbfdc7c, ebp = 0xbfbfe1d8 ---

db> show mount
0xc22f2c94 /dev/ufs/pfsense1 on / (ufs)
0xc22f3000 devfs on /dev (devfs)
0xc22f2a10 /dev/md0 on /tmp (ufs)
0xc22f278c /dev/md1 on /var (ufs)
0xc22f2508 /dev/ufs/cf on /cf (ufs)
0xc22f2000 devfs on /var/dhcpd/dev (devfs)

[moullas]

So.... has this been abandoned or still actively looked into?

Weekend coming up, have time for testing :-D

[jimp]

It hasn't been forgotten, but I haven't had any new leads to try. Ermal thought it might be something that was recently fixed on FreeBSD 8-STABLE, but I wasn't able to make usable patches out of the commits I found so for me that was a dead end (someone else might give it a try)

[Beerman]

Any (good) News? 

[databeestje]

None yet

[tritron]

It seems that the bug still exist interface is very slow unless I execute /etc/rc.conf_mount_rw  in ssh. So workaround is to run /etc/rc.conf_mount_rw  before making any changes

[jimp]

We're still aware it's an issue, there is an open bug in redmine for it.

[phil.davis]

I noticed that Seth Mos has committed "Hopefully tackle the slow nanobsd writes. Redmine ticket #2401" on Github. I guess this is probably built in the version I just updated to:
2.1-BETA0 (i386)
built on Tue Jun 19 20:53:01 EDT 2012
FreeBSD 8.3-RELEASE-p3

Is there any particular sequence of operations that we can/should test to help to see if the issue is resolved or improved?
(I must say, I haven't really noticed the issue on various test nanoBSD installs on my Alix 2D3 in recent weeks!)

[jimp]

It didn't help, unfortunately.

Timing the mount rw then ro calls is sufficient, see the existing redmine ticket for it.

[eweri]

 Just installed snapshot pfSense-2.1-DEVELOPMENT-2g-i386-nanobsd-20120629-1927
still the same problem