Squid and Bandwidth Manager rules
Started by delta
Squid and Bandwidth Manager rules 24 May 2009, 03:48
The question is: If i start forwarding for the clients their http :80 request to the squid cache. what an dhow will the ir set speed limit be handled then?
A: when the page is in the squid cache will it be delivered to them on full lan speed or on the speed limit assigned to their ip address in the separate rule.
b: When the page IS NOT in the squid cache. Squid will go out and fect it for the client at what speed? The speed in the squid rule (most likely) or the speed the original client for whom the page is? And if fetched at speed will it then be delivered to the client?
c: Does the squid rule have to be above the other rules in order to catch the request or not? Meaning if a client rule is below the squid he uses squid redirect and if he is above the squid rule he does not use the squid since his rule is activated or not?
I ask this because some clients use 500kb and others have 1mb subscriptions. Now if it is in the cache it is fine every client receive the cached data on lan speed but if he can get faster internet page download because squid is doing it it defeats the speed limit rules of the bwm, no?
The ideal situation is serving content at fulll speed from the squid cache. But content that has to be retrieved from the net first, has to be handled by BWM.
When the browser requests a page, BWM forwards the request to Squid port 3128. Squid then looks for content in the cache, if there is, the content will not be retrieved from the Internet but instead served from the local cache.
Squid retrieves it's content from the internet side of the computer with BWM installed.
BWM instead schedules traffic on the lan side of the computer.
I would therefore suspect that BWM puts a limit on all traffic, doesn't matter if it is Squid-traffic or Internet traffic. Well, all http traffic is handled by Squid.
Recently, Andrew made a new option available, to allow full speed on the lan.
But if we use this option, all http traffic would just pass BWM as Squid sits on our private lan.
I think it has something to do with the headers of the packets. Please correct me if I'm wrong.
Anyway if someone has more details on this I would be very happy to read it.
Squid and Bandwidth Manager rules 27 May 2009, 23:59
Registered: 12 years ago
Posts: 5 605
One way to enable transparent proxying would be to make one or more rules as explained in this article. Briefly, HTTP traffic from a user gets redirected to the locally installed Squid:
Rate limit: 10000 Protocol: TCP Interface: Internal Source: 192.168.0.10 : Any Destination: Any : 80 Advanced: Process through mapping: SquidThis effectively means the following:
1. The user (192.168.0.10) will get a requested page at 10 KB/s whether the page is in the Squid cache or not.
2. If the requested page is found in the Squid cache, it will be read from there and served to the client at 10 KB/s.
3. If the requested page is not found in the Squid cache, it will retrieve it from the Internet at a full speed, cache it and then serve to the client at 10 KB/s.
Since BWM operates at a low network level, it does not know whether the content was served from the cache or downloaded from the Internet, so it applies a set limit regardless. As a side note, the Ignore LAN traffic option should not be enabled when using Squid as doing so would result in unlimited access to Squid bypassing any limits.
It is possible to limit Squid itself, by making a couple of rules like these:
Rate limit: Unlimited Protocol: TCP Interface: Internal Source: 192.168.0.10 : Any Destination: Any : 80 Advanced: Process through mapping: Squid Rate limit: 100000 Protocol: TCP Interface: External Source: Local host : Any Destination: Any : 80Here the first rule allows unlimited access to Squid, so cached pages will be served at a full LAN speed. The second rule limits Squid's own requests, so if a client requests a not-yet-cached page, it will be served at the rate allowed for Squid to retrieve it or possibly lower one if there are multiple simultaneous requests being made by Squid.
Squid and Bandwidth Manager rules 28 May 2009, 00:17
Registered: 12 years ago
Posts: 5 605
We could make BWM check these markings to allow to serve pages from the cache or from the Internet at different speeds. However, the major problem I see is that neither of these patches is an official part of the Squid core. Therefore, every user would have to apply the patch on his own and compile Squid from the source code (unless the demand is so high, that I would do it myself and publish a binary build )
After the version 2.7 squid includes ZPH in you compilation. (http://squid.acmeconsulting.it/download/squid-2.7.STABLE6-bin.zip)
this is a part of squid.conf.default about the use of ZPH
# TAG: zph_mode
# This option enables packet level marking of HIT/MISS responses,
# either using IP TOS or socket priority.
# off Feature disabled
# tos Set the IP TOS/Diffserv field
# priority Set the socket priority (may get mapped to TOS by OS,
# otherwise only usable in local rulesets)
# option Embed the mark in an IP option field. See also
# See also tcp_outgoing_tos for details/requirements about TOS usage.
# zph_mode off
It will be posible for BWM to recognize content from the cache and content retrieved from the web.
All content served from the cache will be served at lan speed to the user, while content from the web will still be limited.
This is so much better, a great improvement of the service. Can't wait to test it!
commSetTos: FD 17: (40) Function not implemented commSetSocketPriority: FD 13: (40) Function not implemented commSetIPOption: FD 13: (40) Function not implementedI will play with this a little more, and try to compile Squid from the source with these features included.
Update: It seems even though I have enabled BWM to recognize the Squid markings, the problem is that the publicly available Squid build for Windows has got these features disabled. As I have been unable to compile Squid with ZPH enabled, seems there's currently no way to allow instant delivery of cache's contents. Nevertheless, the new BWM feature will be available in further releases. For those interested, this is what it looks like:
Squid and Bandwidth Manager rules 29 May 2009, 00:58
Registered: 11 years ago
Squid and Bandwidth Manager rules 29 May 2009, 01:42
Registered: 11 years ago
No, a Linux system is required.
Acme Consulting S.r.l.
Microsoft Certified Partner
Via Lucia Savarino, 1 10098 - Rivoli (TO) - ITALY
Tel. : +39.011.9530135 Fax. : +39.011.9781115
Da: Lucas Alexandre [mailto:firstname.lastname@example.org]
Inviato: giovedì 28 maggio 2009 16.57
Oggetto: Squid 2.7 for Windows Bug Report
I am from Brazil, sorry my poor english...
Is possible to make SquidNT compatible with ZPH ?
seems the public build of Squid 2.7 (not yet tried 3.0) for Windows does not support ZPH despite the configuration directives. Whether I set zph_mode to tos, priority or option, all failed with these messages in the log:
commSetTos: FD 17: (40) Function not implemented
commSetSocketPriority: FD 13: (40) Function not implemented
commSetIPOption: FD 13: (40) Function not implemented
Andrew, Is there a posibility you will release a Linux version of BWM in the future?
I would recommend not to use the 3.0 version of Squid, there are some bugs present and they have not been resolved since 2005.
It's still an experimental version.
i've installed Squid with BWM, it's works well.
but i want to make password per user basis, i've tried to browse around, and it seems most of the tutorial is based on Linux O/S not under windows.
could anybody help for this issue?...or providing some links, that would be much appreciate...
Thank you for your reply,
but it seems there's misunderstanding.
my installation of squid works with bwm it is already done, and works really fine...(thanks for Andrew).
but now, what i'm trying to do is, to activating authentication for each user which want to access the internet or particular website.
we know that squid has ability to use password for it's security, but i don't know how to achieve that.
and from what i've got from browsing or googling, most of the tutorial for authenticating is for Squid Linux O/S not for Squid Windows, which i have now.
Squid under Linux O/S for example:
1) Create the password file. The name of the password file should be /etc/squid/squid_passwd, and you need to make sure that it's universally readable.
[root@abc tmp]# touch /etc/squid/squid_passwd ==>> Squid Win doesn't have "touch" command
[root@abc tmp]# chmod o+r /etc/squid/squid_passwd
2) Use the htpasswd program to add users to the password file. You can add users at anytime without having to restart Squid. In this case, you add a username called www:
[root@abc tmp]# htpasswd /etc/squid/squid_passwd www
Re-type new password:
Adding password for user www
3) Find your ncsa_auth file using the locate command.
[root@abc tmp]# locate ncsa_auth
i've tried many way, once it done, my squid don't want to start anymore.
please correct me if i'm wrong.
In "squid.conf.default" you can find "auth_param"... but if you enable it, seems you can lose transparent proxing capabilities of squid. You can see in this Warning:
# WARNING: authentication can't be used in a transparently intercepting
# proxy as the client then thinks it is talking to an origin server and
# not the proxy. This is a limitation of bending the TCP/IP protocol to
# transparently intercepting port 80, not a limitation in Squid.
"SquidNT don't support the most of the major features of Squid for Linux." (Guido Serassio - the developer of SquidNT)
I believe that Andrew is working to able BWM to redirect requisitions to squid in another host. (You can redirect connections to port 80 to a other PC with Linux and Squid)
Sorry my english...
Here in Brazil is 3:43AM :o, I go to sleep
i've just found 1 problem that probably you guys can help.
I've tried installed BWM and Squid together in windows server 2008.
BWM works fine, but when i tried to apply squid through BWM (check to enable it), it seems it's slowing down the connection.
so for example i ping-out to the other IP, let say: ping www.google.com -t, before i enable the squid, all are fine, but when i checked or enable the squid, it will become "no resource, and followed by Request time out, and then followed by reply from...and then again request time out OR no resource again".
is anyone has same experience before?, please advise...
ps: i've apply this config & installation on server 2003, totally fine.
at least i just found out why it's happening like that, so if i go to client pc and enter the connection setting, and enable proxy, it will work straight away...so meaning it's not "transparent" anymore, however if i restore the default setting of internet explorer setting...and make it become transparent again, then client pc doesn't work or won't connect to internet anymore.
squid and bwm rules 22 July 2009, 13:53
I am trying to get BWM to redirect normal browsing to a differnt page, Say a disconnect page, how would I go about this. I am running a 2003 with BM in bridge mode. I have installed a web server on the server but I cant seem to get it to work with the port rediredt.
Do i need to install the proxy to get this to work the way I would like it to.