Large size of database/log file NetWorx.db

Started by Alex

Alex

Large size of database/log file NetWorx.db   09 August 2019, 22:29

Hello! I used the forum search and flipped through 20 pages, but could not find the answer to my question.

NetWorx.db database has reached 7.41 GB, and that's too much. I have been using the NetWorx program for 3 years now, but in 2 years the base has grown only to 30 MB. Half a year ago, I turned on the "Ignore local traffic" option, but I did not monitor the size of the database.

I tried to compress the database through the SQLite Viewer, but the database was reduced to 6.62 GB, which is still too much.
According to SQLight Viewer, the most entries are in the "site name" table (23067401 entries) and "siteusage" (114184748 entries).

My question is:
1) How can I reduce the size of the database without losing statistics? For me, the consumption of traffic per day/month is important, detailing on sites is not important. I would like to limit the database to 30-500 MB, no more.
2) Should I disable the "Ignore local traffic" option so that this problem does not recur?

Thanks,
Alex
DonW

Re: Large size of database/log file NetWorx.db   10 August 2019, 23:31

7GB does sound huge. Unfortunately I'm not too computer savvy to be of much help here, but I started using NetWorx many years ago when I needed it to prove that my provider was overcharging me, and my database still is only about 60MB. I do have 'ignore LAN traffic' setting unchecked, so that may be why. I also saw in the manual (this page, at the bottom) that checking that setting is required for per-application data statistics, so it may also be relevant to web sites and your question #2.
SoftPerfect Support forum - Andrew avatar image

Re: Large size of database/log file NetWorx.db   12 August 2019, 13:04

The reason the siteusage and sitename tables are growing so much is that in Ignore local traffic within the LAN mode NetWorx records per IP address statistics. This could add a large number of records if P2P applications like Bittorrent are used, where your computer would connect to many peers for data exchange.

We have just updated NetWorx to deal with this issue: the new builds automatically remove per-site and per-app records older than 3 months. After that, the database is automatically compressed. All usage records other than site/app details are kept intact.

To decrease the size of your database, you can either empty the sitename and siteusage tables manually, or update your installation to the new build, which will compact the database and purge old site/app info for you.
Alex

Re: Large size of database/log file NetWorx.db   17 August 2019, 23:11

Hey DonW, Andrew. Thank you for your responses!

uTorrent is constantly running on my PC. Every day I upload 10-20 gigabytes.

I tried installing a fresh build from pre-release downloads. After starting the program did not respond for about one hour, after that I received the following error:
Exception ESQLiteException in module networx.exe at 000000000374918. TDiskDB: The database file is locked: database is locked.

I completely uninstalled Networks, installed the latest release and upgraded again to a fresh pre-release build. The error was repeated.

So I used SQLite Spy to edit the database.
After deleting the two largest tables, DB takes 1.55 mb. Statistics for day/month did not disappear.
The option "ignore local traffic" I also disabled.

Thank you!
Alex
SoftPerfect Support forum - Ann avatar image
Ann

Re: Large size of database/log file NetWorx.db   19 August 2019, 14:27

Alex, I'm sorry to hear that the preliminary build fix did not fix the issue for you. Database in excess of 7GB is a very large file to process, and it is possible that either you or some process on your computer terminated the optimisation before it completed, causing the error you see.

We will see what other approach we can take for the next release. Your situation is a bit unusual, as NetWorx is normally used by people with limited traffic, to help them watch the limits and check their provider charges. 10-20GB of traffic, especially connection-intensive traffic like torrent, is unusual for NetWorx, and, unfortunately, not well optimised.

Reply to this topic

Sometimes you can get the answer faster if you try the forum search and/or have a look at the software user manual to see if your question has already been answered.

Our forum rules are simple:

  • Be polite.
  • Do not spam.
  • If possible, check your spelling and grammar.

Author:

Email:

Subject

A brief and informative title for your message, approximately 4–8 words:

     

Spam prevention: please enter the following code in the input field below.

 **     **  **     **  **     **  **    **  **    ** 
 ***   ***  ***   ***  **     **  ***   **  **   **  
 **** ****  **** ****  **     **  ****  **  **  **   
 ** *** **  ** *** **  **     **  ** ** **  *****    
 **     **  **     **  **     **  **  ****  **  **   
 **     **  **     **  **     **  **   ***  **   **  
 **     **  **     **   *******   **    **  **    ** 

Message: