Netgate m1n1wall

Author Topic: Varnish help on setting up for multiple internal webservers  (Read 11316 times)

0 Members and 1 Guest are viewing this topic.

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 10004
  • Karma: +5/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #30 on: February 18, 2012, 05:17:53 am »
Remove quotes from your backend Test url

.url = ""/"";

Offline kegler

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #31 on: February 18, 2012, 06:39:48 pm »
omg...it worked now...thank you very much....
« Last Edit: February 18, 2012, 07:18:03 pm by kegler »

Offline canefield

  • Jr. Member
  • **
  • Posts: 86
  • Karma: +0/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #32 on: March 18, 2012, 09:47:23 am »
Each backend means a webserver

inside each configuration, you must provide:
  • Backend name = WindowsServer
  • Ip Address = 192.168.1.4
  • Port = 80 <-- Q
  • Probe URL = "/"(if your server has a valid page on "/")
  • Probe Interval = 5
  • Probe Timeout = 1
  • Probe window = 5
  • Probe Threshold = 3

Q:
Can I enter multiple listening ports e.g. port: "80, 443"? Or should I configure that host several times according to how many ports it is listening on?


Thanks,
Canefield

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 10004
  • Karma: +5/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #33 on: March 18, 2012, 10:09:41 am »
I think you need one port for each config as varnish is going to check backend health as well send traffic based on this port.

Keep in mind That varnish implemets http balance/cache only. If you what to balance https as well, haproxy can do this.

Pound and relayd can do client/firewall ssl comunication and then forward client request to varnish, but there is no gui options for this yet.

Offline Landmine

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #34 on: March 26, 2012, 01:05:26 pm »
Hate to jump on someone else's thread, but I've gone threw all the same motions, but still no results.

web1.domain.com
web2.domain.com

All end up pointing to the pfSense login, so my question is simply this... What IP do you point the Rule to? the pfSense box that host Varnish?

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 10004
  • Karma: +5/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #35 on: March 26, 2012, 01:16:27 pm »
Did you changed  pfsense gui port?
This current varnish gui implementation listens deamon on all interfaces.

Offline Landmine

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #36 on: March 26, 2012, 03:01:49 pm »
I did now and it works!

Guess next time I need to slow down and read all the helpful text.

Thank you

Offline Landmine

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #37 on: March 26, 2012, 05:03:43 pm »
Correction, it was working. I didnt change anything and I'm getting the 503 errors again  :(


------------ Update ------------

I only get the 503 error on 1 of the 2 domains.

------------ Update ------------

Removing the Backend mapping with the http:// on it seemed to fix the issue.

------------ Update ------------

Down again after about 30 seconds.  >:(
« Last Edit: March 26, 2012, 05:39:12 pm by Landmine »

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 10004
  • Karma: +5/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #38 on: March 26, 2012, 06:12:27 pm »
That's probably a backend health url check problem.

Add varnish widget to dashboard to see backend status.

The 503 error means could not find any available backend.

Also try to see backend configuration file on view configuration.

Offline Landmine

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #39 on: March 26, 2012, 09:00:41 pm »
Here is my config

Code: [Select]
# Varnish configuration file
# Automatically generated by the pfSense package system

# This file is located in /var/etc/default.vcl

sub vcl_error {
if (obj.status == 503 && req.restarts < 1) {
restart;
}

set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>"} obj.status " " obj.response {"</title>
<script type="text/javascript">if (top != self) {top.location.href = self.location.href;}</script><script type="text/javascript">var csrfMagicToken = "sid:8caa138ac199fbc97647a8eadbb910361173cee8,1332813414";var csrfMagicName = "__csrf_magic";</script><script src="/csrf/csrf-magic.js" type="text/javascript"></script></head>
<body>
<b>
<p>We are very sorry but an error occurred during this request.</p>
</b>
<p>Please press refresh in your browser to try again.</p>
<p>Varnish Error "} obj.status " " obj.response {"</p>
<p>"} obj.response {"</p>
<h3>Guru Meditation:</h3>
<p>XID: "} req.xid {"</p>
<script type="text/javascript">CsrfMagic.end();</script></body>
</html>"};
return(deliver);

}
       

backend MinecraftBACKEND {
# used in catch_all
.host = "10.0.0.50";
.port = "80";
.first_byte_timeout = 300s;
.connect_timeout = 25s;
.probe = {
.url = "/index.php";
.interval = 5s;
.timeout = 3s;
.window = 5;
.threshold = 3;
}
}



sub vcl_recv {

#BASIC VCL RULES SETTING
#Fix gzip compression
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(gif|jpg|jpeg|bmp|png|ico|img|tga|wmf|gz|tgz|bz2|tbz|mp3|ogg)$") {
unset req.http.Accept-Encoding;
}
else if (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
}
else if (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
}
else {
unset req.http.Accept-Encoding;
}
}
#set client balance identity
set client.identity = client.ip;

#set X-forward
set req.http.X-Forwarded-For = client.ip;


if (req.http.host == "minecraft.domain.com") {
set req.backend = MinecraftBACKEND;
}


#respect client wish to refresh the page
if (req.http.Pragma ~ "no-cache")
{
return(pass);
}

#BASIC VCL RULES ACTIONS
#Disable post cache
if (req.request == "POST") {
return(pass);
}
#Enable static cache
if (req.request=="GET" && req.url ~ "\.(css|js|txt|zip|rtf|flv|swf|html|htm)$") {
return(pass);
}
if (req.request=="GET" && req.url ~ "\.(gif|jpg|jpeg|bmp|png|ico|img|tga|wmf|mp3|ogg)$") {
return(pass);
}
#Disable session cache
if (req.http.Cookie && req.http.Cookie ~ "(PHPSESSID|phpsessid)") {
return(pass);
}
if (req.http.Cookie && req.http.Cookie ~ "(JSESSION|jsession)") {
return(pass);
}
if (req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache") {
return(pass);
}
#Be rfc2616 compliant
if (req.request ~ "^(GET|HEAD|PUT|POST|TRACE|OPTIONS|DELETE)$") {
return(lookup);
}
else {
return(pipe);
}

return(lookup);
}

sub vcl_pipe {

# If we don't set the Connection: close header, any following
# requests from the client will also be piped through and
# left untouched by varnish. We don't want that.
set req.http.connection = "close";
# Note: no "pipe" action here - we'll fall back to the default
# pipe method so that when any changes are made there, we
# still inherit them.

}

sub vcl_fetch {

#Disable cache when backend is starting a session
if (beresp.http.Set-Cookie && beresp.http.Set-Cookie ~ "(PHPSESSID|phpsessid)") {
return(pass);
}
if (beresp.http.Set-Cookie && beresp.http.Set-Cookie ~ "(JSESSION|jsession)") {
return(pass);
}
if (beresp.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache") {
return(pass);
}

# Varnish respects the wishes of the backend application.
if (beresp.http.Pragma ~ "no-cache" || beresp.http.Cache-Control ~ "(no-cache|no-store|private)") {
return(pass);
}
    ## If the request to the backend returns a code other than 200, restart the loop
    ## If the number of restarts reaches the value of the parameter max_restarts,
    ## the request will be error'ed.  max_restarts defaults to 4.  This prevents
    ## an eternal loop in the event that, e.g., the object does not exist at all.
    if (beresp.status != 200 && beresp.status != 403 && beresp.status != 404 &&
    beresp.status != 303 && beresp.status != 302 && beresp.status != 301 && beresp.status != 401 ) {
restart;
        }

   

return(deliver);
}

sub vcl_deliver {
##set resp.http.X-Served-By = server.hostname;
  if (obj.hits > 0) {
    set resp.http.X-Cache = "HIT";
    set resp.http.X-Cache-Hits = obj.hits;
  } else {
    set resp.http.X-Cache = "MISS";
  }
return(deliver);
}


Offline marcelloc

  • Hero Member
  • *****
  • Posts: 10004
  • Karma: +5/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #40 on: March 26, 2012, 09:09:18 pm »
can you try to access backend test url?

http://10.0.0.50/index.php

Also, what you get on dashboard varnish widget?

Offline Landmine

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #41 on: March 26, 2012, 09:11:39 pm »
The URL works get when I'm at home and here is a pic of what the widget looks like.





Thank you for taking the time to help me.

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 10004
  • Karma: +5/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #42 on: March 26, 2012, 09:20:01 pm »
Varnish could not get this url from your web server.

Can you try an static html, a full http request or something else?


Offline Landmine

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #43 on: March 26, 2012, 09:32:22 pm »
So far so good. Now if I wanted to add more servers to the list, I'd basically copy the settings and just change the Backend name, IP and Mapping?

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 10004
  • Karma: +5/-0
    • View Profile
Re: Varnish help on setting up for multiple internal webservers
« Reply #44 on: March 26, 2012, 11:07:57 pm »
Yes.  :)

Just don't forget to check backend status on Dashboard.