Delay in launch when the database contains a lot of data

Started by Eco-Drive

Eco-Drive

Delay in launch when the database contains a lot of data   28 February 2020, 14:20

Hello,

I have a problem: I cannot launch NetWorx after changing "Keep per site/application records for (days)" option.

Presently my database is ~1.7GB (already compressed by networx, before that it was ~6GB) containing ~7 months worth of data.

After upgrading to the current version of Networx, I set the hidden option "Keep per site/application records for (days)" to a relative small setting (24 days) and closed networx.

But after that, networx would not start normally anymore, the icon did not appear in taskbar. The process networx.exe did appear in taskmgr but there is no disk activity. To my guessing after altering the new hidden option, networx should trim the database. Now it just sits in the background doing nothing, apparently in some form of deadlock.

Also I have a new suggestion for the new automatic trimming feature: if the new hidden option is set to zero (keep data forever) then do not perform the scanning and compressing on program launch because for those who have a large database, this could take literally hours. From an old user's perspective, his PC would suddenly be thrashing the hard disk for hours on each boot after upgrading networx for no apparent reason.

Thanks!
SoftPerfect Support forum - Andrew avatar image

Re: Delay in launch when the database contains a lot of data   28 February 2020, 15:06

It would not be on each boot. If there is no data older than N days, or N = 0, it will simply start without doing anything.

Though, in your case it is probably deleting old data. Try opening the database in any SQLite manager and executing:
DELETE FROM appname;
DELETE FROM sitename;
DELETE FROM appusage;
DELETE FROM siteusage;
DELETE FROM appsiteusage;
All usage data figures will stay, just without the apps and sites details. Then compress the database (VACUUM).

This should start everything over and allow you to open NetWorx.
Hi Andrew,

Thank you very much for the timely response!

Quote

It would not be on each boot. If there is no data older than N days, or N = 0, it will simply start without doing anything.


Well on my computer, before I modified "KeepSiteInfoForDays", it would definitely take about 45 minutes for each program launch, during which I can see the size of "Networx.db-wal" slowly increases to the size of Networx.db, and disk usage locked to 100%.

Maybe there was already some corruption with my database, such as some key responsible for tracking the age of data not being set properly?

Quote

Though in your case it's probably deleting old data.

After I modified the hidden option to 24 days, the next program launch it did not look like it was doing anything - in resource monitor the process shows zero disk activity. I waited for a few hours before killing the process in taskmgr.

I wonder if the database already have some issue, causing networx to scan and compress the database on every launch when "KeepSiteInfoForDays = 0" and freeze without doing anything when "KeepSiteInfoForDays = 24"

Thanks.
SoftPerfect Support forum - Andrew avatar image

Re: Delay in launch when the database contains a lot of data   28 February 2020, 16:45

If so, the database probably does have some issues. You can safely delete the database file and a new valid database will be created (but all settings and usage data will be lost).
Thanks for the advice. The DELETE FROM command ran for more than 24 hours before I ran out of patience. I ended up creating a new database and copied over the usage table.
Dear Networx developers,
Thanks for continuously monitoring the forum and provide support.

I want to add that the performance of "Keep app/site data for days" option needs improving. Currently, when this feature is enabled, program startup time is too slow even with a very small database.

For example, Networx will take about 3 minutes to start for a freshly created database with just half day of data (~7MB), which I think is beginning to fall into "unusable" category. A novice user might just assume the program is defective. Are there any possible remedies for this? Or is the performance of sqlite just not enough?

Thanks!

Reply to this topic

Sometimes you can find a solution faster if you try the forum search, have a look at the knowledge base, or check the software user manual to see if your question has already been answered.

Our forum rules are simple:

  • Be polite.
  • Do not spam.
  • Write in English. If possible, check your spelling and grammar.

Author:

Email:

Subject

A brief and informative title for your message, approximately 4–8 words:

     

Spam prevention: please enter the following code in the input field below.

 **      **  **    **  **    **   *******   **     ** 
 **  **  **  ***   **  ***   **  **     **   **   **  
 **  **  **  ****  **  ****  **  **           ** **   
 **  **  **  ** ** **  ** ** **  ********      ***    
 **  **  **  **  ****  **  ****  **     **    ** **   
 **  **  **  **   ***  **   ***  **     **   **   **  
  ***  ***   **    **  **    **   *******   **     ** 

Message: