I suspect the device you use to save your config file needs to have a FreeBSD file system on it. Most USB sticks seem to come with FAT file system (not FreeBSD). I haven't used floppies for a long time but I suspect they also come per-formatted with FAT or come unformatted.
I expect if you search the forum or the pfSense Wiki you'll find a recipe for appropriate formatting. If you can't find one there its worth looking in the m0n0wall documentation - follow the link at
http://m0n0.ch/wall/ (pfSense is a fork of m0n0wall.)
An easier option might be to use the web GUI: Diagnostics -> Backup/Restore to backup the configuration file to another system over the network.