pfSense Support Subscription

Author Topic: Possible DHCP Issues  (Read 11296 times)

0 Members and 1 Guest are viewing this topic.

Offline AsgardianFW

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +2/-1
    • View Profile
Possible DHCP Issues
« on: February 21, 2016, 08:48:51 pm »
I haven't paid any attention to my DHCP lease statuses in several weeks, but recently I had need to look at them again and noticed that not all devices (that usually show a hostname) are showing their hostname anymore.  I only have 3 devices on my network that don't provide a hostname in the DHCP handshake and I'm not talking about those.  I have about 6 out of 25 "normal" devices that are no longer showing the hostname in Status / DHCP Leases (and as a consequence, their names are not resolvable by IP).  I know that several weeks ago, all of these devices were correctly showing their hostname and were resolvable via DNS as well (IP to hostname).    In the DHCP system log, the DHCPREQUEST and DHCPACK entries for these devices sometimes show the hostname and sometimes not.  I'm not sure if this is relevant, but the only change I've made to pfSense's configuration during this time is that I have swapped the LAN and WAN interfaces (both in pfSense configuration and physical network cables) and I upgrade snapshots about once a week.  I'm currently on 2.3.b.20160216.1417.  Any thoughts on the issue would be appreciated.  I run some reports that rely on converting LAN IP's to hostnames and those without DHCP registered hostnames are obviously not converting.

Offline cmb

  • Hero Member
  • *****
  • Posts: 11230
  • Karma: +893/-7
    • View Profile
    • Chris Buechler
Re: Possible DHCP Issues
« Reply #1 on: February 22, 2016, 08:20:46 pm »
It pulls those from the /var/dhcpd/var/db/dhcpd.leases file, which is populated by dhcpd when the client sends a hostname in its request. Sounds like the clients in question aren't sending their hostname.

Offline AsgardianFW

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +2/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #2 on: February 22, 2016, 08:55:21 pm »
I do believe the clients in question are sending their hostname as some of the clients have been around a while and have sent their hostnames reliably in the past.  Today, I turned off 3 of the devices and deleted their DHCP reservation.  When the devices were turned back on, everything worked OK.  They received an IP address and pfSense received the hostnames.

On a different but related note, pfSense only allows deleting reservations when it thinks the device is "offline".  Once a reservation is deleted, it immediately marks all other devices as offline and their reservations can then be deleted.  Waiting a few minutes and refreshing will once again show some of the devices as online again.

Offline cmb

  • Hero Member
  • *****
  • Posts: 11230
  • Karma: +893/-7
    • View Profile
    • Chris Buechler
Re: Possible DHCP Issues
« Reply #3 on: February 23, 2016, 05:25:38 pm »
If you have them as static mappings, you must specify the hostname in the mapping.

The online/offline status there is whether or not that IP exists in the ARP cache.

Offline yon

  • Sr. Member
  • ****
  • Posts: 397
  • Karma: +4/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #4 on: February 24, 2016, 12:22:11 am »
yesterday, PF DHCP happen stop work,  I have update to 2.3-BETA (amd64)
built on Mon Feb 22 05:50:51 CST 2016  seem work now.
If you are interested in free peering for clearnet and dn42,contact me !

Offline AsgardianFW

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +2/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #5 on: February 24, 2016, 08:53:08 am »
I'm not talking about the static mappings.  Those have always worked correctly.

I mentioned the bit about about offline/online status because I didn't think deleting a reservation would change the ARP cache (and maybe it really doesn't) but immediately after deleting a non-static reservation, everything is offline in the DHCP reservations list until you do a refresh a short time later.

Offline AsgardianFW

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +2/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #6 on: February 25, 2016, 08:47:49 am »
Some additional info on this topic.  To help understand what is happening here, I've been focusing just on my cell phone as it is one of the devices that has inconsistent hostnames in DHCP reservation list.  Following is the sequence of events:
1) Last night I turned WiFi off on my phone until it appeared offline in pfSense and I deleted its DHCP reservation
2) At 23:27, I turned WiFi back on and it immediately obtained an IP address and the hostname was correctly shown in DHCP
3) I left the phone on overnight in the office and at 7:15 the following day, the hostname was gone from pfSense, but the IP is correct and is online and I even used the phone to access the internet between 7:00 and 7:15.
4) At 7:23, I moved to the other side of the office (where the phone would have moved onto a different access point).  There was another DHCP request as a result of the access point move and suddenly the hostname is back.

Following is the DHCP log entries regarding this device:
Code: [Select]
Feb 25 07:23:31 dhcpd DHCPACK on 10.11.12.224 to 24:da:3b:a9:5f:1b (android-dc49b48ab4ff452) via re1
Feb 25 07:23:31 dhcpd DHCPREQUEST for 10.11.12.224 from 24:da:3b:a9:5f:1b via re1
Feb 24 23:27:35 dhcpd DHCPACK on 10.11.12.224 to 24:da:3b:a9:5f:1b via re1
Feb 24 23:27:35 dhcpd DHCPREQUEST for 10.11.12.224 from 24:da:3b:a9:5f:1b (android-dc49b48ab4ff452) via re1
Feb 24 23:27:07 dhcpd DHCPACK on 10.11.12.224 to 24:da:3b:a9:5f:1b (android-dc49b48ab4ff452) via re1
Feb 24 23:27:07 dhcpd DHCPREQUEST for 10.11.12.224 from 24:da:3b:a9:5f:1b via re1

Do remember that prior to a few weeks ago, I never had this problem on the beta and I've never had this problem while the 2.2 series.  I am curious if I'm the only one seeing this problem.  If so, what would cause the DHCP server to drop the hostname when the log isn't showing any communication from the time it got the hostname to the time it dropped it.  My DHCP lease time is 24 hours, so the phone wouldn't normally attempt any further DHCP communication until 11:27 the next day...so the lack of communication seems correct.

Offline divsys

  • Hero Member
  • *****
  • Posts: 919
  • Karma: +91/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #7 on: February 25, 2016, 11:57:17 am »
Quote
My DHCP lease time is 24 hours, so the phone wouldn't normally attempt any further DHCP communication until 11:27 the next day

Small aside - many (most?) devices will attempt to re-establish their DHCP lease well ahead of it's expiry.
It's not uncommon to see a lease attempt at 1/3 to 1/2 the expiry time.

That may affect your view of the relevant logs.
$.02
-jfp

Offline AsgardianFW

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +2/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #8 on: February 25, 2016, 01:01:27 pm »
divsys, you are correct.  I have ascertained that my phone will refresh its lease at 1/2 time which is why I didn't expect anything until 11:37.  My phrasing may have been unclear.  The point I was trying to make is that there was no communication between the phone and DHCP server (and nor was there any expected), but somehow the DHCP server dropped the hostname.  I'm going to test this theory later, but I am guessing that the hostname will reappear at the 1/2 time renegotiation.  I'm also going to track more closely exactly when the DHCP server drops the hostname to see if the timing correlates with anything.

Also, no one else is chiming in reporting similar issues, so I'm left to wonder if it's just me.  Or perhaps the default shorter lease doesn't show the problem as often.  If no one else can confirm, then perhaps I should reimage my box from the latest snap to see if the problem goes away.  This is a relatively minor problem, but it makes our reports harder to read because it shows IP addresses instead of names.  MarksiPhone is clear.  10.11.12.210 is unclear.

Offline cmb

  • Hero Member
  • *****
  • Posts: 11230
  • Karma: +893/-7
    • View Profile
    • Chris Buechler
Re: Possible DHCP Issues
« Reply #9 on: February 25, 2016, 01:45:37 pm »
That log snippet shows it is sending a hostname, "android-dc49b48ab4ff452". What do its lease entries in /var/dhcpd/var/db/dhcpd.leases look like?

Offline AsgardianFW

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +2/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #10 on: February 25, 2016, 02:49:46 pm »
The entries are consistent with what I see in the web gui.  When the gui shows the hostname, there is a hostname line for that device in the file.  When the gui doesn't show the hostname, the file does not have a hostname line.  Currently, the hostname is still in there (but neither me nor my phone have been there since around 7:30 this morning):
Code: [Select]
lease 10.11.12.224 {
  starts 4 2016/02/25 12:23:31;
  ends 5 2016/02/26 12:23:31;
  cltt 4 2016/02/25 12:23:31;
  binding state active;
  next binding state free;
  rewind binding state free;
  hardware ethernet 24:da:3b:a9:5f:1b;
  uid "\001$\332\073\251\137\033";
  set vendor-class-identifier = "dhcpcd-5.5.6";
  client-hostname "android-dc49b48ab4ff452";
}


Offline cmb

  • Hero Member
  • *****
  • Posts: 11230
  • Karma: +893/-7
    • View Profile
    • Chris Buechler
Re: Possible DHCP Issues
« Reply #11 on: February 25, 2016, 03:48:54 pm »
So that shows dhcpd is using what's given to it, and dhcpleases process is using what's available to it. Has to be the client in that case, or I guess it's feasible some intermediate device is manipulating the DHCP requests though that seems unlikely. Packet capture DHCP requests to verify.


Offline AsgardianFW

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +2/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #12 on: February 25, 2016, 08:38:44 pm »
It will take me 2-5 days to get the packet capture as I have to go to another office for a few days.  Are you interested in all DHCP (which will be very verbose as we have quite a few Windows machine and they seem to speak DHCP way too often) or are you just interested in my phone's chatter?

Offline cmb

  • Hero Member
  • *****
  • Posts: 11230
  • Karma: +893/-7
    • View Profile
    • Chris Buechler
Re: Possible DHCP Issues
« Reply #13 on: February 25, 2016, 09:13:10 pm »
Just the requests from the phone would suffice, but it might be hard to filter strictly on that. I'd probably filter with just 'port 67 or port 68' on tcpdump, even across days of DHCP requests with hundreds of clients, that wouldn't add up to anything really significant. Something like the following via SSH:

Code: [Select]
tcpdump -i igb0 -s 0 -w dhcp.pcap port 67 or port 68
where igb0 is the LAN NIC. ctrl-c to stop that after you've replicated the problem, then you can open that dhcp.pcap file in Wireshark and clearly see whether or not it's sending a client-hostname.

Offline AsgardianFW

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +2/-1
    • View Profile
Re: Possible DHCP Issues
« Reply #14 on: February 28, 2016, 07:42:55 pm »
OK.  I started a packet capture when pfSense was showing hostnames for all my devices (except the ones I know don't provide hostnames).  Just 15 minutes into the packet capture and pfSense "lost" the hostnames for 2 devices.  Only 18 DHCP packets were captured and I was able to study each one carefully (in Wireshark).  For the 2 devices that lost their hostname, there were no DHCP packets that referenced the devices MAC address or IP address.

Note this was in the DHCP log a few minutes into the packet capture:
Code: [Select]
Feb 28 20:18:37 dhcpd Wrote 22 leases to leases file.
Feb 28 20:18:37 dhcpd Wrote 0 new dynamic host decls to leases file.
Feb 28 20:18:37 dhcpd Wrote 0 deleted host decls to leases file.
.
.
Feb 28 20:22:20 dhclient Creating resolv.conf
Feb 28 20:22:20 dhclient RENEW
« Last Edit: February 28, 2016, 07:58:34 pm by AsgardianFW »