SoftPerfect Bandwidth Manager

Speed limit account for packet headers too?

Started by shunt010

Speed limit account for packet headers too?   09 July 2007, 05:28

Hi all.

Here's an interesting one...

I'm running a wireless network and using BWM to limit the bandwidth.

Fine, until someone comes along with P2P.

I've not got a big problem with P2P, except it appears to send a lot of very empty packets (20 bytes or less).

This puts a lot of load on the network because BWM doesn't include the packet headers and the extra packet overhead on the preambles.

Would it be possible to add to BWM so that you can include packet headers in the calculation of data transferred/bandwidth, and also put a "fixed" header length on too - a 802.11g header is equivalent to around 100bytes of data I reckon, so I'd like to add 100 to the length of each packet, so those users who insist on sending a lot of empty packets don't get such good throughput as if they sent fewer fully loaded packets.


This way the system wouldn't grind to a halt under the load of users sending 1000's of "empty" packets a second, BWM would reward users who are using it sensibly with "full" packets and punish those who are just sending 1000's of empty packets a second.

Thoughts?
SoftPerfect Support forum - Andrew avatar image

Speed limit account for packet headers too?   10 July 2007, 06:05

Well, good question. Actually BWM deals with the following headers for each packet: Ethernet (14 bytes: MAC source, MAC destination, 2-byte protocol type), IPv4 (20 bytes), TCP (20 bytes). So each packet is at least of 54 bytes. When calculating used bandwidth BWM includes header size, for example if a TCP packet would contain 5 bytes of data, the word 'HELLO', the actually counted length would be 59 bytes. If you consider, adding 100 bytes to each packet may be a solution, we can publish a custom build with that feature to see how it really affects the situation.

Speed limit account for packet headers too?   19 September 2007, 23:30

Hi Andrew.

Sorry about the delay in getting back to you.

Looking at the problem in more detail, it appears that adding a fixed number of bytes on to the packet size for the Wifi header would work very well. We're getting nearly 70% loading on the Wifi side of things from 1 user who's sending UDP packets with around 3 bytes payload. But there's around 10000 packets a second going through from what I can see, which with the wifi header is putting a very large loading on the system, although when it becomes TCP/IP it's very small because it doesn't have the preamble.

Also is it possible that the queue size per rule could be limited? Even with the queue size at 10000, around 9994 are being used up by just one rule, meaning that other rules are experiencing a lot of dropped packets when they shouldn't be. A small queue size per rule but a large one overall might be useful, if it's possible?

This way when the amount of the queue used by a particular rule gets too high, it drops packets for that rule rather than using up any more of the queue?

Speed limit account for packet headers too?   29 November 2007, 01:46

Hi Andrew.

Did you get anywhere with this?

Trying adding on predefined extra amounts for all packets would be very interesting for me. We're having a lot of problem with packet flooding from P2P on Wireless networks, and blocking P2P isn't a solution.

What we need to do is to add a predefined number of bytes onto each packet (the predefined number of bytes I need to try to determine), so if lots of empty packets start coming through, the bandwidth manager holds them off.

At the moment we're getting in excess of 8000 packets with less than 100 bytes payload coming through a second on a user who's supposed to be limited to around 200kbits/sec. The actual data is less than 200kbits/sec, but they're putting 100% loading on the wireless because of the 802.11 packet headers.

If I could somehow get the BWM to penalize heavy packet flow (such as adding 1000 bytes onto each packet recieved, then setting the rate to be correct to give the right throughput assuming the packets are loaded to the MTU), then I think this would really help, so then the customers with lots of empty packets will get very poor throughput.
Samuel Hunt

Speed limit account for packet headers too?   31 December 2007, 00:09

Hi Andrew

Any ideas if you plan to implement this or not?

Please?
SoftPerfect Support forum - Andrew avatar image

Speed limit account for packet headers too?   04 January 2008, 01:33

Yes. Finally we are ready to do this. Some questions need to be clarified though.

Quote

Also is it possible that the queue size per rule could be limited? Even with the queue size at 10000, around 9994 are being used up by just one rule, meaning that other rules are experiencing a lot of dropped packets when they shouldn't be.


Actually when you specify the queue size, it refers to the per rule setting. If you specify 100, this means every rule will have a queue of up to 100 packets. If one of the queues overflows, this does not affect the others. If you set this buffer too large, e.g. 10000, that means every rule may hold up to 10000 packets in the queue which in no doubt affects performance.

Quote

If I could somehow get the BWM to penalize heavy packet flow (such as adding 1000 bytes onto each packet received, then setting the rate to be correct to give the right throughput assuming the packets are loaded to the MTU), then I think this would really help, so then the customers with lots of empty packets will get very poor throughput.


I think the mechanism needs to be a little bit sophisticated than just adding a fixed number of bytes to each packet. Normally, packets in a network are 1500 bytes MTU or less. There is no sense in adding anything to large packets. For example, if the user downloads a file, there is a flow of packet sized 1500 bytes each. If BWM would add a fixed number of bytes on these packets, this would result an overall reduction in throughput. It's reasonable to add a fixed number of bytes to smaller packets, but what do you think could other criteria for packets to be penalized besides their size? May be some IP/TCP flags or data content?
Samuel Hunt

Speed limit account for packet headers too?   12 January 2008, 09:29

Hi Andrew,

Thanks for the reply.

I do remember reading somewhere now about the queuing mechanism. I've found that a small queue size works better, at first I thought it was due to the queue being shared between all rules, but I think now it's because it stops protocols keep sending data at too high a rate and causing network congestion before the protocol realises it's hit the limit. Running a lower queue length seems to work better because packets get dropped far earlier so TCP/IP and suchlike realises that it's reached the bandwidth limit and behaves much more gracefully without congesting the entire link.

How I invisaged it working was to add a fixed number of bytes to each packet, then compensate for this when it knows the overall bandwidth. I might have my maths wrong, but I can get a packet of 1500 bytes (including headers) through my network, so the MTU is 1500. With 1500 byte long packets, 10 packets a second gives just under 15kbytes/sec.

If the fixed number of bytes was, for example, 1500 bytes per packet, this would then mean that the throughput would drop to 7.5kbits/sec. However, if the BWM took this into the calculation when it loads the rules, it knows the theoretical maximum number of packets that could go through without the addition, then adds the extra bit on, it would give my rule 30kbytes/sec bandwidth, giving a throughput assuming fully loaded packets of 15kbytes/sec.

Assuming though that the user was going to run with some of these very empty packets, which is tying up the network, such as 30 bytes, then this would allow roughly 20 packets a second, giving a throughput of 600 bytes/sec, which would then penalize the user heavily for running "empty" packets, but it would reduce the load off the system caused by sending these "empty" packets.


Only a thought of how I guess it would work. Obviously 1500 bytes is a lot to add on to each packet, and probably around 300 bytes would be far more representative of the actual overhead of the 802.11 protocol, but it would mean that users using these protocols can't any longer bring the network to a standstill just by running UDP filesharing packets with just 20 bytes payload (I've seen approximately 1000 of these packets per second at times, which gave an actual throughput of something like 20kbytes/sec, and slowed everyone else's experience down).

Anyway, that's what I found, hopefully it's of some help and sorry about pestering you like this about it!
SoftPerfect Support forum - Andrew avatar image

Speed limit account for packet headers too?   14 January 2008, 23:27

We have just uploaded a test build. Once you install it and start the service at least once, there will be two parameters in the file SERVICE.INI located in the installation folder. These parameters are read on startup, so you have to stop the service, change them, and start the service again if conducting an experiment.

These two parameters are PenalityThreshold and PenalityAddition. Both should be in the range 1 to 1500. If either of them is zero, no user will be penalised. The algorithm is simple. If the packet size is less than PenalityThreshold, it will be padded with PenalityAddition bytes. Therefore, you can play with it to find values suitable for your goals.

If you have any questions or comments, feel free to ask.
Samuel Hunt

Speed limit account for packet headers too?   17 January 2008, 04:51

Hi Andrew.

Initial results are very promising.

I decided to try your approach of only penalising small packets, so set both to 500. It appears to be working from what I can see so far...

Out of interest, what is the combined checkbox on the bandwidth setting?
SoftPerfect Support forum - Andrew avatar image

Speed limit account for packet headers too?   19 January 2008, 17:40

Hi Samuel,

The "combined" checkbox allows to set a shared limit for uploads and downloads. Please refer to this thread for details.

Speed limit account for packet headers too?   20 January 2008, 20:47

Hi Andrew,

I see where you're heading with the notifications tab now, and the alert manager. Nice! I like the idea of emailing to a user address when quota reached or limit changed etc... Very nice.

Perhaps there should be a sticky subject that contains just the latest build version information for testing only, or something along those lines?

Cheers,
Gavin.
SoftPerfect Support forum - Andrew avatar image

Speed limit account for packet headers too?   20 January 2008, 23:30

I think most users are interested in well-tested production releases, not intermediate builds. However, sounds like a good idea to make a sticky thread for announcements about these builds if case if someone wants new functionality immediately. Such a thread is to be created closer to the new version release which can be expected within a month.

Speed limit account for packet headers too?   24 January 2008, 00:44

Hi Andrew.

I've been testing this for a while now.

It's working very well. Network congestion has gone down to practically nil, P2P is being hammered but everything else seems good.

It's perhaps more effective than the "real" P2P thing that's in BWM.

BTW, any ideas about dynamic bandwidth allocation, have you had any more thoughts on that, or packet prioritization?
Lukas

This version work in Kerio Winroute ?   17 February 2008, 02:44

Andrew, this special version work in kerio winroute ?
Lukas

Kerio Winroute   17 February 2008, 03:10

Andrew,

New Kerio Winroute version 6.4.x, have a network driver protocol in interface manager.

I installed the normal build of BWM, and have no problem. This is recomended ?

Speed limit account for packet headers too?   28 February 2008, 11:38

One thought with this....

Would it be possible to enable on a per rule or per interface basis?

I've recently started to experiment with bandwidth limiting both interfaces on my bridge, so ease congestion on the ADSL line.

Very good results - 2 rules (upstream and downstream) that are set on the ADSL side, to ensure the ADSL line doesn't congest (the router seems to handle this poorly, simply queuing up to 1000 packets. This gives very poor latency (4000ms +).

Using BWM for the upstream throttling gives excellent latency on pings even at full saturation (15ms to first hop), giving the impression of no saturation to the customer. On actual packets, it drops packets as necessary, which does a much better job of keeping the latency down and managing the bandwidth better, giving throughput latency of around 100ms when saturated, which is lots better than the 4000ms + I was getting previously.

However, I worry that small packets, which are an issue on Wifi but not on ADSL, may be causing the upstream and downstream not to reach 100% loading. Whilst this is an excellent feature on the Wifi interface, on the ADSL interface it is not desirable.

Any thoughts?
SoftPerfect Support forum - Andrew avatar image

Speed limit account for packet headers too?   03 March 2008, 21:28

I think it is a good idea to make the penalty configurable for each rule. I will post an update when you can download the file for testing purposes.

Speed limit account for packet headers too?   07 March 2008, 22:52

Hi Andrew,

This would be fantastic, look forward to trying it...
SoftPerfect Support forum - Andrew avatar image

Speed limit account for packet headers too?   13 March 2008, 17:09

The update is ready for testing. There is no installation package at this time. In order to evaluate it, please do the following:

1. Download this file.
2. Unpack the relevant files from the archive over the existing installation (and close the GUI and stop the service before you replace the files).
3. Run the service and GUI.

You'll see the penalty options now can be configured using GUI (in 'Options') and there is a check box when you create or alter a rule that allows to enable/disable penalty imposed on the rule.

Speed limit account for packet headers too?   14 March 2008, 09:15

Mind if I ask what might the Quota section in the options be for in the future?


Gavin.
SoftPerfect Support forum - Andrew avatar image

Speed limit account for packet headers too?   14 March 2008, 11:33

Global quota settings such as first day of the month and first day of the week have been moved there. However, please do not consider this build as something to be released. A number of other features, which are not included in this build, will be available in a release candidate.
This topic has been closed.