Delay in launch when the database contains a lot of data
6 posts · Closed
Hello,
I have a problem: I cannot launch NetWorx after changing the "Keep per site/application records for (days)" option.
Presently my database is ~1.7GB (already compressed by NetWorx, before that it was ~6GB) containing ~7 months' worth of data.
After upgrading to the current version of NetWorx, I set the hidden option "Keep per site/application records for (days)" to a relatively small setting (24 days) and closed NetWorx.
But after that, NetWorx would not start normally anymore; the icon did not appear in the taskbar. The process networx.exe did appear in taskmgr, but there is no disk activity. To my guessing, after altering the new hidden option, NetWorx should trim the database. Now it just sits in the background doing nothing, apparently in some form of deadlock.
Also, I have a new suggestion for the new automatic trimming feature: if the new hidden option is set to zero (keep data forever), then do not perform the scanning and compressing on program launch because, for those who have a large database, this could take literally hours. From an old user's perspective, his PC would suddenly be thrashing the hard disk for hours on each boot after upgrading NetWorx for no apparent reason.
Thanks!
It would not be on each boot. If there is no data older than N days, or N = 0, it will simply start without doing anything.
Though, in your case it is probably deleting old data. Try opening the database in any SQLite manager and executing:
DELETE FROM appname; DELETE FROM sitename; DELETE FROM appusage; DELETE FROM siteusage; DELETE FROM appsiteusage;
All usage data figures will stay, just without the apps and sites details. Then compress the database (VACUUM).
This should start everything over and allow you to open NetWorx.
Hi Andrew,
Thank you very much for the timely response!
It would not be on each boot. If there is no data older than N days, or N = 0, it will simply start without doing anything.
Well on my computer, before I modified "KeepSiteInfoForDays", it would definitely take about 45 minutes for each program launch, during which I can see the size of "Networx.db-wal" slowly increases to the size of Networx.db, and disk usage locked to 100%.
Maybe there was already some corruption with my database, such as some key responsible for tracking the age of data not being set properly?
Though in your case it's probably deleting old data.
After I modified the hidden option to 24 days, the next program launch it did not look like it was doing anything - in resource monitor the process shows zero disk activity. I waited for a few hours before killing the process in taskmgr.
I wonder if the database already have some issue, causing networx to scan and compress the database on every launch when "KeepSiteInfoForDays = 0" and freeze without doing anything when "KeepSiteInfoForDays = 24"
Thanks.
If so, the database probably does have some issues. You can safely delete the database file and a new valid database will be created (but all settings and usage data will be lost).
Thanks for the advice. The DELETE FROM command ran for more than 24 hours before I ran out of patience. I ended up creating a new database and copied over the usage table.
Dear Networx developers,
Thanks for continuously monitoring the forum and provide support.
I want to add that the performance of "Keep app/site data for days" option needs improving. Currently, when this feature is enabled, program startup time is too slow even with a very small database.
For example, Networx will take about 3 minutes to start for a freshly created database with just half day of data (~7MB), which I think is beginning to fall into "unusable" category. A novice user might just assume the program is defective. Are there any possible remedies for this? Or is the performance of sqlite just not enough?
Thanks!