Netgate SG-1000 microFirewall

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Topics - PiBa

Pages: [1]
Development / Building pfSense 2.2 / FreeBSD 10.1 - WORKING STEPS
« on: November 05, 2014, 04:00:08 pm »
These are the steps i followed to get to a bootable & installable pfSense .iso file:
If you find anything wrong, or have faster/shorter/better way of doing it, please let me know :D.

Code: [Select]
How to build: -- pfSense 2.2 FreeBSD 10.1-PRERELEASE --

### 0 - Gain access to the pfsense-tools ###
Use PuTTYgen to generate a "SSH-2 RSA" key and Export the OpenSSH key to a file locally for usage later.
also save the 'ssh-rsa ..... key-YYYYMMDD' public key to fill in the online registration form.

Information about how the signup works also here:
-Sign the ICLA or CCLA agreement online
-then also sign the LA, and configure your public ssh key, that will gaining you access to the pfsense-tools repository after a few minutes.

#### my VirtualBox build machine configuration ####
 disk : 20GB
 vcpu : 4  << more could be faster..
 mem  : 3 GB is enough to use 'memory disks' while compiling which speeds up the build process, if you have plenty more probably wont hurt ;)
- download and install into the virtual machine :
-install: lib & src
-enable: ssh
-configure user: admin

## allow root to login over ssh / for easy access with winscp and putty (NEVER DO THIS on a production machine, only for testing/easy access)
echo PermitRootLogin yes >> /etc/ssh/sshd_config
service sshd restart

Login with WinSCP to be able to easely transfer files from/to the FreeBSD build machine.
- copy your earlier saved ssh private key to the system using WinSCP to: /home/admin/private_putty_key_ssh.ppk

### 1. Begin pfSense bootstrap by running these shell commands:

echo "WITHOUT_X11=yo" > /etc/make.conf
echo "OPTIONS_UNSET=X11" >> /etc/make.conf
echo "BATCH=yo" >> /etc/make.conf
mkdir -p /home/pfsense/pfSenseGITREPO /usr/pfSensesrc

portsnap fetch extract
cd /usr/ports/devel/git && make depends install

### 2. Load SSH key and checkout sources for pfSense-tools
eval `ssh-agent -c`
chmod 600 /home/admin/private_putty_key_ssh.ppk
ssh-add /home/admin/private_putty_key_ssh.ppk

cd /home/pfsense && git clone tools
-Are you sure you want to continue connecting (yes/no)? yes   << required to type all 3 letters of y e s

### 3. Set correct publicly available repositories
## 'pfmechanics' is used by ESF internally for fast local mirrors of several repositories to speed up their build process it is not available from outside.
## p.s. dont try to use git@ for github, it wil fail later because it will try to use the ssh key loaded above..

sed -i "" "s,,," /usr/home/pfsense/tools/builder_scripts/
sed -i "" "s,,," /usr/home/pfsense/tools/builder_scripts/
sed -i "" "s,,," /usr/home/pfsense/tools/builder_scripts/
sed -i "" "s,,," /usr/home/pfsense/tools/builder_scripts/

## to check if the above sed commands went ok, the command below should show nothing:
cat /usr/home/pfsense/tools/builder_scripts/ | grep

### 4. Checkout Freesbie2
cd /home/pfsense && git clone git:// freesbie2

### 5. set version to build
cd /home/pfsense/tools/builder_scripts
./ RELENG_2_2

### 6. Rebuild the bsdinstaller.
cd /home/pfsense/tools/builder_scripts
scripts/ ; scripts/

### 7. Build ports
### Now it will also be checking out FreeBSD sources, this will take quite some time with no visible progress, in the background the 'git' process will take a little cpu usage..
### tmp file in /usr/pfSensesrc/src/.git/objects/pack will slowly grow over 800MB..

cd /home/pfsense/tools/builder_scripts
./ --build-pfPorts --no-cleanrepos

## to check progress see what ports will be build in: /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/buildports.RELENG_2_2
## It should show all ports build without failures..
  >>> Ports with failures:        0
  >>> Building tools/crytpo...Done!
  ==> End of pfPorts...
  >>> Operation ./ has ended at Wed Nov  5 22:16:57 CET 2014

### 7. Build pfSense iso
cd /home/pfsense/tools/builder_scripts
./ iso --no-cleanrepos

>>> ISO created: Wed Nov  5 22:53:02 CET 2014
>>> NOTE: waiting for jobs:  to finish...
>>> /tmp/pfSense/ now contains:
total 239208
drwxr-xr-x   2 root  wheel   512B Nov  5 22:53 .
drwxrwxrwt  10 root  wheel   512B Nov  5 22:52 ..
-rw-r--r--   1 root  wheel   234M Nov  5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso
-rw-------   1 root  wheel     0B Nov  5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso.gz
>>> Operation ./ has ended at Wed Nov  5 22:53:02 CET 2014

### 8. YOUR DONE (hopefully) ###
The resulting iso should show
ls -l /tmp/pfSense/

   -rw-r--r--  1 root  wheel  90629317 Nov  5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso.gz

### p.s. ###
Im using --no-cleanrepos flag above to speedup second and further build attempts(the same day?) by skipping the slow checkout of FreeBSD sources.

#### LOGS ####
  Kernel buildworld log:
  ports building (failure) logs (empty is good, during building there can be temporary logfiles here):

#### To apply a custom patch written against FreeBSD code this is one way to apply it ####
 - add patch file divert-reply10.1.patch to : /usr/home/pfsense/tools/patches/releng/10.1
 - add the patch to the list of patches, depending on the format of the patch git/diff you might need to specify it a little different.:
    echo "-p1~~divert-reply10.1.patch~" >> /usr/home/pfsense/tools/builder_scripts/conf/patchlist/releng/10.1/patches

It seams ARP packets not using CARP-MAC cause problems when trying to work around that..

Here is my situation:
Ive got 2 PFsense boxes (2.0.1-RELEASE) which im trying to configure for a lot of purposes.
# These are my current requirements:
-failover if 1 internet connection fails
-failover if 1 PFsense server fails
# failover is supposed to include:
-LAN client machines trying to use the internet using gateway CARP-LAN ip
-roadWarior's using OpenVPN, connecting to externIP of router(s), portforwarding to internal CARP-WAN ip
Also by use of VLAN's webservers accessible from the internet with portforwarding are separated from the LAN by firewall rules.. should the webserver become compromised the other LAN machines would not be accessible.
# If all fails, be able to change the gateways of the client machines and then without any other reconfiguring be able to use the internet.

internet is connected behind 2 modem/router-devices both have internet IPs and a internal IPs( and And have a portforwarding rule to the CARP-WAN-IP UDPport 96 which is used for OpenVPN trafic.
pfSense also uses 2 these devices as it gateway's.
PFsense.3  LAN-ip  WAN-ip: [00:0c:29:a1:cc:db]  (backup)
PFsense.5  LAN-ip  WAN-ip: [00:0c:29:46:26:2d]  (master)
CARP-WAN: [00:00:5e:00:01:02]
Yes i know WAN and LAN interfaces on the same subnet/network is pretty strange, but seams to best fit some of the current requirements and constraints..

It seams to work 'mostly'.. If PFsense.5 is running i can connect with OpenVPN client and it works 'OK', after shutting down PF.5 and failing over to PF.3 after reconnection of OpenVPN clients all seams fine.
Now after the PF.5 comes back up and takes back the MASTER of the carp interfaces. my Draytek modem/router keeps recieving udp packets from PF.3 interface and source and as because of this assumes the PF.3 WAN interface is the way to send packets to destined for Also other connections destined for go to box PF.3 because of a wrong ARP-cache.

TCPdump from PF.5
Code: [Select]
22:20:01.679471 00:50:7f:c9:ac:b0 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has tell, length 46
22:20:01.679477 00:0c:29:46:26:2d > 00:50:7f:c9:ac:b0, ethertype ARP (0x0806), length 42: Reply is-at 00:00:5e:00:01:02, length 28
TCPdump from PF.3
Code: [Select]
22:20:00.688137 00:0c:29:a1:cc:db > 00:50:7f:c9:ac:b0, ethertype IPv4 (0x0800), length 591: > 89.98.X.X.1194: UDP, length 549
22:20:00.713112 00:50:7f:c9:ac:b0 > 00:0c:29:a1:cc:db, ethertype IPv4 (0x0800), length 143: 89.98.X.X.1194 > UDP, length 101
22:20:01.346365 00:0c:29:a1:cc:db > 00:50:7f:c9:ac:b0, ethertype IPv4 (0x0800), length 271: > 89.98.X.X.1194: UDP, length 229
22:20:01.502634 00:50:7f:c9:ac:b0 > 00:0c:29:a1:cc:db, ethertype IPv4 (0x0800), length 191: 89.98.X.X.1194 > UDP, length 149

After clearing ARP-cache of the draytek it asks again what it the destination for, and gets a reply as can be seen above.. However after that the UDP packets from PF.3 still come in and seam to 'overwrite' the ARPcache. Have as a result that the OpenVPN client still is/gets connected to PF.3 ..

The draytek can be configured to not allow reconfiguring by 'spoofed' packets using telnet  IP ARP ALLOW , but then also rejects the ARP-Reply from PF.5, because it is not send from interface 00:00:5e:00:01:02..

With telnet to Draytek ther is an option "ip arp accept" which can be used for setting this option: "Ethernet source address doesn't match ARP sender address. Accept illegal ARP REPLY packets or not." This is currently configured to accept the Reply from PF.5 even though packet sender address does not match..

So i think my main question is, is it possible to have the ARP traffic from a CARP-ip also leave with the CAPR interface MAC-address? I think it would solve this issue.

Pages: [1]