Squid and Bandwidth Manager rules

Started by delta

delta

Squid and Bandwidth Manager rules   24 May 2009, 03:48

I installed bwm and squid.
The question is: If i start forwarding for the clients their http :80 request to the squid cache. what an dhow will the ir set speed limit be handled then?
A: when the page is in the squid cache will it be delivered to them on full lan speed or on the speed limit assigned to their ip address in the separate rule.
b: When the page IS NOT in the squid cache. Squid will go out and fect it for the client at what speed? The speed in the squid rule (most likely) or the speed the original client for whom the page is? And if fetched at speed will it then be delivered to the client?
c: Does the squid rule have to be above the other rules in order to catch the request or not? Meaning if a client rule is below the squid he uses squid redirect and if he is above the squid rule he does not use the squid since his rule is activated or not?

I ask this because some clients use 500kb and others have 1mb subscriptions. Now if it is in the cache it is fine every client receive the cached data on lan speed but if he can get faster internet page download because squid is doing it it defeats the speed limit rules of the bwm, no?

squid and bwm rules   25 May 2009, 15:16

Good question. I would like to hear some opinions on this too.
The ideal situation is serving content at fulll speed from the squid cache. But content that has to be retrieved from the net first, has to be handled by BWM.

When the browser requests a page, BWM forwards the request to Squid port 3128. Squid then looks for content in the cache, if there is, the content will not be retrieved from the Internet but instead served from the local cache.

Squid retrieves it's content from the internet side of the computer with BWM installed.
BWM instead schedules traffic on the lan side of the computer.
I would therefore suspect that BWM puts a limit on all traffic, doesn't matter if it is Squid-traffic or Internet traffic. Well, all http traffic is handled by Squid.

Recently, Andrew made a new option available, to allow full speed on the lan.
But if we use this option, all http traffic would just pass BWM as Squid sits on our private lan.

I think it has something to do with the headers of the packets. Please correct me if I'm wrong.
Anyway if someone has more details on this I would be very happy to read it.

Cheers,
Alacran.

squid and bwm rules   25 May 2009, 23:24

Hi guys... Sorry my bad english smile

Squid have a feature called "Zero Penault Hit - ZPH". Seek on the google for this (my english is unable to explain better).

Maybe Andrew know what to do work ZPH with BWM.
SoftPerfect Support forum - Andrew avatar image

Squid and Bandwidth Manager rules   27 May 2009, 23:59

Well, the whole thing depends on how the actual rules are made to handle traffic from to Squid. Let us consider the most common scenario: <Users> - <Server with BWM and Squid> - <Router/Modem>. In this scenario BWM and and Squid both sit on the same computer.

One way to enable transparent proxying would be to make one or more rules as explained in this article. Briefly, HTTP traffic from a user gets redirected to the locally installed Squid:
Rate limit: 10000
Protocol: TCP
Interface: Internal
Source: 192.168.0.10 : Any
Destination: Any : 80
Advanced: Process through mapping: Squid
This effectively means the following:
1. The user (192.168.0.10) will get a requested page at 10 KB/s whether the page is in the Squid cache or not.
2. If the requested page is found in the Squid cache, it will be read from there and served to the client at 10 KB/s.
3. If the requested page is not found in the Squid cache, it will retrieve it from the Internet at a full speed, cache it and then serve to the client at 10 KB/s.

Since BWM operates at a low network level, it does not know whether the content was served from the cache or downloaded from the Internet, so it applies a set limit regardless. As a side note, the Ignore LAN traffic option should not be enabled when using Squid as doing so would result in unlimited access to Squid bypassing any limits.

It is possible to limit Squid itself, by making a couple of rules like these:
Rate limit: Unlimited
Protocol: TCP
Interface: Internal
Source: 192.168.0.10 : Any
Destination: Any : 80
Advanced: Process through mapping: Squid

Rate limit: 100000
Protocol: TCP
Interface: External
Source: Local host : Any
Destination: Any : 80
Here the first rule allows unlimited access to Squid, so cached pages will be served at a full LAN speed. The second rule limits Squid's own requests, so if a client requests a not-yet-cached page, it will be served at the rate allowed for Squid to retrieve it or possibly lower one if there are multiple simultaneous requests being made by Squid.
SoftPerfect Support forum - Andrew avatar image

Squid and Bandwidth Manager rules   28 May 2009, 00:17

As Lucas pointed to Zero Penalty Hit patch for SQUID (ZPH), this is something interesting. The basic idea is fairly simple: if a page is served from the cache, that is a cache HIT occurred, Squid signals this by marking IP packets in the connection serving this page. This could be done by altering a 8-bit field called Type of Service in an IPv4 packet, or by adding an extra option to the packet as it is made in various patch implementations.

We could make BWM check these markings to allow to serve pages from the cache or from the Internet at different speeds. However, the major problem I see is that neither of these patches is an official part of the Squid core. Therefore, every user would have to apply the patch on his own and compile Squid from the source code (unless the demand is so high, that I would do it myself and publish a binary build smile )

squid and bwm rules   28 May 2009, 08:40

Hi Andrew!

After the version 2.7 squid includes ZPH in you compilation. (http://squid.acmeconsulting.it/download/squid-2.7.STABLE6-bin.zip)

this is a part of squid.conf.default about the use of ZPH

# TAG: zph_mode
# This option enables packet level marking of HIT/MISS responses,
# either using IP TOS or socket priority.
# off Feature disabled
# tos Set the IP TOS/Diffserv field
# priority Set the socket priority (may get mapped to TOS by OS,
# otherwise only usable in local rulesets)
# option Embed the mark in an IP option field. See also
# zph_option.
#
# See also tcp_outgoing_tos for details/requirements about TOS usage.
#
#Default:
# zph_mode off

Thanks!!
SoftPerfect Support forum - Andrew avatar image

squid and bwm rules   28 May 2009, 12:07

Hi Lucas,

Thanks for pointing this. Indeed Squid 2.7 and above does include ZPH. Therefore, I presume I can make BWM recognize these markings. I will publish a test version shortly.

Andrew.

squid and bwm rules   28 May 2009, 15:54

Yeah! That would be absolutely fantastic!
It will be posible for BWM to recognize content from the cache and content retrieved from the web.
All content served from the cache will be served at lan speed to the user, while content from the web will still be limited.
This is so much better, a great improvement of the service. Can't wait to test it!

Cheers,
Arthur.
SoftPerfect Support forum - Andrew avatar image

squid and bwm rules   28 May 2009, 16:09

Well, I have implemented that in BWM, but it seems the public build of Squid 2.7 (not yet tried 3.0) for Windows does not support ZPH despite the configuration directives. Whether I set zph_mode to tos, priority or option, all failed with these messages in the log:
commSetTos: FD 17: (40) Function not implemented
commSetSocketPriority: FD 13: (40) Function not implemented
commSetIPOption: FD 13: (40) Function not implemented
I will play with this a little more, and try to compile Squid from the source with these features included.

Update: It seems even though I have enabled BWM to recognize the Squid markings, the problem is that the publicly available Squid build for Windows has got these features disabled. As I have been unable to compile Squid with ZPH enabled, seems there's currently no way to allow instant delivery of cache's contents. Nevertheless, the new BWM feature will be available in further releases. For those interested, this is what it looks like:

[image]

Squid and Bandwidth Manager rules   29 May 2009, 00:58

I sent a email to ACME Consulting asking if possible make squid compatible with ZPH. I am waiting the answer.

Squid and Bandwidth Manager rules   29 May 2009, 01:42

The answer:

Hi,

No, a Linux system is required.

Regards

Guido Serassio
Acme Consulting S.r.l.
Microsoft Certified Partner
Via Lucia Savarino, 1 10098 - Rivoli (TO) - ITALY
Tel. : +39.011.9530135 Fax. : +39.011.9781115
Email: guido.serassio@acmeconsulting.it
WWW: [www.acmeconsulting.it]

Da: Lucas Alexandre [mailto:lukalexandre@yahoo.com.br]
Inviato: giovedì 28 maggio 2009 16.57
A: squidnt@acmeconsulting.it
Oggetto: Squid 2.7 for Windows Bug Report

Hi,

I am from Brazil, sorry my poor english...

Is possible to make SquidNT compatible with ZPH ?

seems the public build of Squid 2.7 (not yet tried 3.0) for Windows does not support ZPH despite the configuration directives. Whether I set zph_mode to tos, priority or option, all failed with these messages in the log:

commSetTos: FD 17: (40) Function not implemented

commSetSocketPriority: FD 13: (40) Function not implemented

commSetIPOption: FD 13: (40) Function not implemented

Thanks!

squid and bwm rules   29 May 2009, 12:24

Well, here comes the obvious:
Andrew, Is there a posibility you will release a Linux version of BWM in the future?

I would recommend not to use the 3.0 version of Squid, there are some bugs present and they have not been resolved since 2005.
It's still an experimental version.

Cheers,
Arthur.
SoftPerfect Support forum - Andrew avatar image

squid and bwm rules   29 May 2009, 12:45

Hi Arthur, no this does not seem possible in any foreseeable future. However, I think I could be able to make BWM to be able to redirect connections to an external host (not only to the local machine as it is now). In that case, the problem might be solved by having a Linux box, either real or virtual doing caching.

squid and bwm rules   29 May 2009, 23:18

Redirect connections to an external host (in my case I will use wmware) is a good solution for this...

squid and bwm rules   09 July 2009, 14:58

Hi all,
i've installed Squid with BWM, it's works well.

but i want to make password per user basis, i've tried to browse around, and it seems most of the tutorial is based on Linux O/S not under windows.

could anybody help for this issue?...or providing some links, that would be much appreciate...

Thanks guys!


Cheers!

squid and bwm rules   09 July 2009, 15:55

SoftPerfect now has this tutorial: [www.softperfect.com]

squid and bwm rules   09 July 2009, 16:17

Hi Lucas,
Thank you for your reply,
but it seems there's misunderstanding.

my installation of squid works with bwm it is already done, and works really fine...(thanks for Andrew).
but now, what i'm trying to do is, to activating authentication for each user which want to access the internet or particular website.
we know that squid has ability to use password for it's security, but i don't know how to achieve that.
and from what i've got from browsing or googling, most of the tutorial for authenticating is for Squid Linux O/S not for Squid Windows, which i have now.

Squid under Linux O/S for example:
1) Create the password file. The name of the password file should be /etc/squid/squid_passwd, and you need to make sure that it's universally readable.

[root@abc tmp]# touch /etc/squid/squid_passwd ==>> Squid Win doesn't have "touch" command
[root@abc tmp]# chmod o+r /etc/squid/squid_passwd

2) Use the htpasswd program to add users to the password file. You can add users at anytime without having to restart Squid. In this case, you add a username called www:

[root@abc tmp]# htpasswd /etc/squid/squid_passwd www
New password:
Re-type new password:
Adding password for user www
[root@abc tmp]#

3) Find your ncsa_auth file using the locate command.

[root@abc tmp]# locate ncsa_auth
/usr/lib/squid/ncsa_auth
[root@abc tmp]#

i've tried many way, once it done, my squid don't want to start anymore.

please correct me if i'm wrong.

Thank you.

squid and bwm rules   09 July 2009, 16:43

Hi Asylum,

In "squid.conf.default" you can find "auth_param"... but if you enable it, seems you can lose transparent proxing capabilities of squid. You can see in this Warning:

# WARNING: authentication can't be used in a transparently intercepting
# proxy as the client then thinks it is talking to an origin server and
# not the proxy. This is a limitation of bending the TCP/IP protocol to
# transparently intercepting port 80, not a limitation in Squid.

"SquidNT don't support the most of the major features of Squid for Linux." (Guido Serassio - the developer of SquidNT)

I believe that Andrew is working to able BWM to redirect requisitions to squid in another host. (You can redirect connections to port 80 to a other PC with Linux and Squid)

Sorry my english...

Here in Brazil is 3:43AM :o, I go to sleep smile

squid and bwm rules   12 July 2009, 10:31

Asylum,

What you want to do is possible, with Radius autentication. Radius also runs on Windows (Winradius). Hope this helps.

Cheers,
Arthur.

squid and bwm rules   14 July 2009, 15:44

Hi All,

i've just found 1 problem that probably you guys can help.

I've tried installed BWM and Squid together in windows server 2008.
BWM works fine, but when i tried to apply squid through BWM (check to enable it), it seems it's slowing down the connection.
so for example i ping-out to the other IP, let say: ping www.google.com -t, before i enable the squid, all are fine, but when i checked or enable the squid, it will become "no resource, and followed by Request time out, and then followed by reply from...and then again request time out OR no resource again".

is anyone has same experience before?, please advise...

Thank you.

ps: i've apply this config & installation on server 2003, totally fine.

squid and bwm rules   16 July 2009, 15:41

Hi All,

at least i just found out why it's happening like that, so if i go to client pc and enter the connection setting, and enable proxy, it will work straight away...so meaning it's not "transparent" anymore, however if i restore the default setting of internet explorer setting...and make it become transparent again, then client pc doesn't work or won't connect to internet anymore.

any thought?

Thank you.
Leoco

squid and bwm rules   22 July 2009, 13:53

HI,

I am trying to get BWM to redirect normal browsing to a differnt page, Say a disconnect page, how would I go about this. I am running a 2003 with BM in bridge mode. I have installed a web server on the server but I cant seem to get it to work with the port rediredt.

Do i need to install the proxy to get this to work the way I would like it to.

Thanks

Leo

Sometimes you can get the answer faster if you try the forum search and/or have a look at the software user manual to see if your question has already been answered.

Our forum rules are simple:

  • Be polite.
  • Do not spam.
  • If possible, check your spelling and grammar.

Author:

Email:

Subject

A brief and informative title for your message, approximately 4–8 words:

     

Spam prevention: please enter the following code in the input field below.

 ********  **     **  ********  ********  **      ** 
 **    **   **   **   **        **    **  **  **  ** 
     **      ** **    **            **    **  **  ** 
    **        ***     ******       **     **  **  ** 
   **        ** **    **          **      **  **  ** 
   **       **   **   **          **      **  **  ** 
   **      **     **  **          **       ***  ***  

Message: