The pfSense Store

Author Topic: Reverse squid proxy does not show always tomcat webpage  (Read 2457 times)

0 Members and 1 Guest are viewing this topic.

Offline klokslag

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
Reverse squid proxy does not show always tomcat webpage
« on: June 27, 2012, 09:38:18 am »
Hi,

I use wonna use reverse squid proxy before our Novell Vibe server.
Vibe is a Tomcat Application that uses internal 8080 and 8443.

The config now is:

vibe.domain.ext --> 80   --> squid --> 8080 --> Vibe server
vibe.domain.ext --> 443 --> squid --> 8443 --> Vibe server

With the ssl config i use an cert on the squid en one on Vibe.
I made two webservers an two mappings in the config.

The problem is that the Vibe site does not always show up. Its not stable. Its hanging or very very slow.
I also use Groupwise and that goes right or just a simple site. I also set the firewall open, but no results

Who can help me with this problem or what do i wrong. Is there a log that i can read except the realtime log?

Pleas help!

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 9996
  • Karma: +4/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #1 on: June 27, 2012, 10:14:18 am »
check squid access.log and cache.log

Offline klokslag

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #2 on: June 27, 2012, 12:58:31 pm »
Thanks for your reply.

In the access log the http site does get logged correct one time.
The second time nothing is logged. When i call the https one line get logged with a time out i gues.

Quote
1340818529.227      3 82.168.159.164 TCP_MISS/302 291 GET http://vibe.domain.ext/ - FIRST_UP_PARENT/HOST1 -
1340818529.289      7 82.168.159.164 TCP_MISS/302 536 GET http://vibe.domain.ext/ssf/a/ - FIRST_UP_PARENT/HOST1 -
1340818529.365     25 82.168.159.164 TCP_MISS/200 30574 GET http://vibe.domain.ext/ssf/a/c/p_name/ss_forum/p_action/1/action/view_permalink/entityType/user/entryId/ss_user_id_place_holder/showWhatsNew/1/vibeonprem_url/1/vibeonprem_root/1 - FIRST_UP_PARENT/HOST1 text/html
1340818529.940      3 82.168.159.164 TCP_MISS/200 6416 GET http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtteaming.nocache.js? - FIRST_UP_PARENT/HOST1 text/javascript
1340818530.229      1 82.168.159.164 TCP_MISS/404 1621 GET http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/tiny_mce_3_2_7/tiny_mce/plugins/preelementfix/langs/nl.js? - FIRST_UP_PARENT/HOST1 text/html
1340818531.125     60 82.168.159.164 TCP_MISS/200 727 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.233     10 82.168.159.164 TCP_MISS/200 564 GET http://vibe.domain.ext/ssf/a/do? - FIRST_UP_PARENT/HOST1 text/html
1340818531.267     51 82.168.159.164 TCP_MISS/200 646 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.346     68 82.168.159.164 TCP_MISS/200 629 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.472     57 82.168.159.164 TCP_MISS/200 620 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.475     49 82.168.159.164 TCP_MISS/200 646 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.501     66 82.168.159.164 TCP_MISS/200 871 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.688     53 82.168.159.164 TCP_MISS/200 642 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.698     56 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.703     53 82.168.159.164 TCP_MISS/200 657 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.813     58 82.168.159.164 TCP_MISS/200 669 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.843     58 82.168.159.164 TCP_MISS/200 626 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.955     51 82.168.159.164 TCP_MISS/200 605 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818591.828     71 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818651.783     63 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818726.244  59270 82.168.159.164 TCP_MISS/000 0 GET https://vibe.domain.ext/ - FIRST_UP_PARENT/HOST2 -


In the access log i get two errors that FD22 an FD24 already in use is.
I think that the adresses are defined two time?

Quote
2012/06/27 19:32:11| Starting Squid Cache version 3.1.20 for amd64-portbld-freebsd8.1...
2012/06/27 19:32:11| Process ID 2482
2012/06/27 19:32:11| With 11095 file descriptors available
2012/06/27 19:32:11| Initializing IP Cache...
2012/06/27 19:32:11| DNS Socket created at [::], FD 11
2012/06/27 19:32:11| DNS Socket created at 0.0.0.0, FD 12
2012/06/27 19:32:11| Adding domain annamaria.nl from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 127.0.0.1 from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 194.151.228.18 from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 172.16.2.22 from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 172.16.2.20 from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 194.151.228.34 from /etc/resolv.conf
2012/06/27 19:32:11| User-Agent logging is disabled.
2012/06/27 19:32:11| Referer logging is disabled.
2012/06/27 19:32:11| Unlinkd pipe opened on FD 17
2012/06/27 19:32:11| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
2012/06/27 19:32:11| Store logging disabled
2012/06/27 19:32:11| Swap maxSize 102400 + 8192 KB, estimated 8507 objects
2012/06/27 19:32:11| Target number of buckets: 425
2012/06/27 19:32:11| Using 8192 Store buckets
2012/06/27 19:32:11| Max Mem  size: 8192 KB
2012/06/27 19:32:11| Max Swap size: 102400 KB
2012/06/27 19:32:11| Version 1 of swap file with LFS support detected...
2012/06/27 19:32:11| Rebuilding storage in /var/squid/cache (DIRTY)
2012/06/27 19:32:11| Using Least Load store dir selection
2012/06/27 19:32:11| Current Directory is /etc
2012/06/27 19:32:11| Loaded Icons.
2012/06/27 19:32:11| helperOpenServers: Starting 0/0 'ssl_crtd' processes
2012/06/27 19:32:11| helperOpenServers: No 'ssl_crtd' processes needed.
2012/06/27 19:32:11| Accepting  HTTP connections at 172.16.0.23:8090, FD 20.
2012/06/27 19:32:11| Accepting  accelerated HTTP connections at xx.xx.xx.xx:80, FD 21.
2012/06/27 19:32:11| commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
2012/06/27 19:32:11| Accepting HTTPS connections at xx.xx.xx.xx:443, FD 23.
2012/06/27 19:32:11| commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use
2012/06/27 19:32:11| Accepting ICP messages at [::]:7, FD 25.
2012/06/27 19:32:11| HTCP Disabled.
2012/06/27 19:32:11| Configuring Parent 172.16.2.48/8080/0
2012/06/27 19:32:11| Configuring Parent 172.16.2.48/8443/0
2012/06/27 19:32:11| Ready to serve requests.
2012/06/27 19:32:11| Done reading /var/squid/cache swaplog (263 entries)
2012/06/27 19:32:11| Finished rebuilding storage from disk.
2012/06/27 19:32:11|       263 Entries scanned
2012/06/27 19:32:11|         0 Invalid entries.
2012/06/27 19:32:11|         0 With invalid flags.
2012/06/27 19:32:11|       263 Objects loaded.
2012/06/27 19:32:11|         0 Objects expired.
2012/06/27 19:32:11|         0 Objects cancelled.
2012/06/27 19:32:11|         0 Duplicate URLs purged.
2012/06/27 19:32:11|         0 Swapfile clashes avoided.
2012/06/27 19:32:11|   Took 0.03 seconds (9953.83 objects/sec).
2012/06/27 19:32:11| Beginning Validation Procedure
2012/06/27 19:32:11|   Completed Validation Procedure
2012/06/27 19:32:11|   Validated 551 Entries
2012/06/27 19:32:11|   store_swap_size = 742
2012/06/27 19:32:12| storeLateRelease: released 0 objects

So i still cannot find what i'am doing wrong. Can you help me more?
This is my squid config:
Quote
# This file is automatically generated by pfSense
# Do not edit manually !
http_port 172.16.0.23:8090
icp_port 7

pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_default_language nl
icon_directory /usr/local/etc/squid/icons
visible_hostname localhost
cache_mgr admin@localhost
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
sslcrtd_children 0
logfile_rotate 14
shutdown_lifetime 3 seconds
# Allow local network(s) on interface(s)
acl localnet src  172.16.0.0/16
uri_whitespace strip

# Break HTTP standard for flash videos. Keep them in cache even if asked not to.
refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

# Let the clients favorite video site through with full caching
acl youtube dstdomain .youtube.com
cache allow youtube
cache_mem 8 MB
maximum_object_size_in_memory 32 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir ufs /var/squid/cache 100 16 256
minimum_object_size 0 KB
maximum_object_size 4 KB
offline_mode offcache_swap_low 90
cache_swap_high 95
acl donotcache dstdomain "/var/squid/acl/donotcache.acl"
cache deny donotcache   
# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:    1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0  0%  0
refresh_pattern .    0  20%  4320
# No redirector configured


#Remote proxies


# Setup some default acls
acl allsrc src all
acl localhost src 127.0.0.1/32
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 85 3128 1025-65535
acl sslports port 443 563 85
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT

acl allowed_subnets src 172.16.0.0/16 172.17.20.0/24 172.17.30.0/24 172.17.40.0/24 172.17.50.0/24 172.17.60.0/24 172.17.70.0/24 172.17.80.0/24
http_access allow manager localhost
 
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
http_access allow localhost

quick_abort_min -1 KB
quick_abort_max 0 KB
request_body_max_size 0 KB
reply_body_max_size 4194304 KB allsrc
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
# Throttle extensions matched in the url
acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
delay_access 1 allow throttle_exts
delay_access 1 deny allsrc

# Reverse Proxy settings
http_port xx.xx.xx:80 accel defaultsite=vibe.domain.ext vhost
https_port xx.xx.xx.xx:443 accel cert=/usr/local/etc/squid/4fe475cabe502.crt key=/usr/local/etc/squid/4fe475cabe502.key defaultsite=vibe.domain.ext
http_port xx.xx.xx.xx:80 accel defaultsite=vibe.domain.ext vhost
https_port xx.xx.xx.xx:443 accel cert=/usr/local/etc/squid/4fe475cabe502.crt key=/usr/local/etc/squid/4fe475cabe502.key defaultsite=vibe.domain.ext

#
cache_peer 172.16.2.48 parent 8080 0 proxy-only no-query originserver login=PASS name=HOST1

#
cache_peer 172.16.2.48 parent 8443 0 proxy-only no-query originserver login=PASS ssl sslflags=DONT_VERIFY_PEER front-end-https=auto name=HOST2

acl URI1 url_regex -i http://vibe.domain.ext/.*$
acl URI2 url_regex -i https://vibe.domain.ext/.*$
cache_peer_access HOST1 allow URI1
cache_peer_access HOST2 allow URI2
cache_peer_access HOST1 deny allsrc
cache_peer_access HOST2 deny allsrc
never_direct allow URI1
never_direct allow URI2
http_access allow URI1
http_access allow URI2

deny_info TCP_RESET allsrc

# Custom options


# Setup allowed acls
# Allow local network(s) on interface(s)
http_access allow allowed_subnets
http_access allow localnet
# Default block all to be sure
http_access deny allsrc


Offline marcelloc

  • Hero Member
  • *****
  • Posts: 9996
  • Karma: +4/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #3 on: June 27, 2012, 01:26:10 pm »
Quote
2012/06/27 19:32:11| commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
2012/06/27 19:32:11| commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use

Did you changed pfsense gui to other port then 80 and 443 at system -> advanced?

Offline klokslag

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #4 on: June 27, 2012, 01:46:21 pm »
That is correct. I have set it on port 85.

The 4 lines are the same external ip address but FD differents.
What does FD means?

Quote
Accepting  accelerated HTTP connections at xx.xx.xx.xx:80, FD 21.
commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
Accepting HTTPS connections at xx.xx.xx.xx:443, FD 23.
commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use

Offline klokslag

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #5 on: June 27, 2012, 03:29:40 pm »
When i use only the http reverse mode the Novell Vibe site is stable reachable.

As soon as i also use the https reverse mode the connection wil become unstable. This is also when i only use the https reverse mode. The site wil become unstable to connect.

Do you have an other idea that i can check?

Offline klokslag

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #6 on: June 28, 2012, 06:17:02 am »
Has anyone else an idea? :'(

Offline klokslag

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #7 on: July 07, 2012, 01:22:33 pm »
Solved! With the new package marcelloc made and the right regex its works good. Thanx! :D

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 9996
  • Karma: +4/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #8 on: July 07, 2012, 03:20:46 pm »
Solved! With the new package marcelloc made and the right regex its works good. Thanx! :D

Thanks for the feedback klokslag. Did it worked with both protocols(http and https)?

att,
Marcello Coutinho

Offline klokslag

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #9 on: July 08, 2012, 02:51:26 am »
It works with both protocols. I'm trying now an second url to work with https. But haven't succeeded yet. Because it still redirect to THE default/first site. Http works fine.

Do you have an idea.

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 9996
  • Karma: +4/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #10 on: July 08, 2012, 09:46:04 am »
Do you have an idea.

Do you see any erros/ missing config on squid.conf file?

Just a note that may help:
I'm finishing a new package gui version for apache+mod_security, I think this new package will do a much better job on http/https reverse proxy.
I've tried to finish it last week but there is still missing features to include.

att,
Marcello Coutinho

Offline klokslag

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
Re: Reverse squid proxy does not show always tomcat webpage
« Reply #11 on: July 08, 2012, 12:33:55 pm »
I do not see any errors in the cache logn the or squid.conf.
I think it has to do with the default site with the https protocol.
When i put there the second url in it, that url works too but not the first.

But if you think apche+mod_security will work better? I will wait when you finished it and will test it.
Let me now when your finished it.

Regards.