All Forums
> NetWorx
> Current topic
Delay in launch when the database contains a lot of data
Started by Eco-Drive
Eco-Drive
Delay in launch when the database contains a lot of data 28 February 2020, 14:20 |
Hello,
I have a problem: I cannot launch NetWorx after changing "Keep per site/application records for (days)" option.
Presently my database is ~1.7GB (already compressed by networx, before that it was ~6GB) containing ~7 months worth of data.
After upgrading to the current version of Networx, I set the hidden option "Keep per site/application records for (days)" to a relative small setting (24 days) and closed networx.
But after that, networx would not start normally anymore, the icon did not appear in taskbar. The process networx.exe did appear in taskmgr but there is no disk activity. To my guessing after altering the new hidden option, networx should trim the database. Now it just sits in the background doing nothing, apparently in some form of deadlock.
Also I have a new suggestion for the new automatic trimming feature: if the new hidden option is set to zero (keep data forever) then do not perform the scanning and compressing on program launch because for those who have a large database, this could take literally hours. From an old user's perspective, his PC would suddenly be thrashing the hard disk for hours on each boot after upgrading networx for no apparent reason.
Thanks!
I have a problem: I cannot launch NetWorx after changing "Keep per site/application records for (days)" option.
Presently my database is ~1.7GB (already compressed by networx, before that it was ~6GB) containing ~7 months worth of data.
After upgrading to the current version of Networx, I set the hidden option "Keep per site/application records for (days)" to a relative small setting (24 days) and closed networx.
But after that, networx would not start normally anymore, the icon did not appear in taskbar. The process networx.exe did appear in taskmgr but there is no disk activity. To my guessing after altering the new hidden option, networx should trim the database. Now it just sits in the background doing nothing, apparently in some form of deadlock.
Also I have a new suggestion for the new automatic trimming feature: if the new hidden option is set to zero (keep data forever) then do not perform the scanning and compressing on program launch because for those who have a large database, this could take literally hours. From an old user's perspective, his PC would suddenly be thrashing the hard disk for hours on each boot after upgrading networx for no apparent reason.
Thanks!
Re: Delay in launch when the database contains a lot of data 28 February 2020, 15:06 |
Admin Registered: 18 years ago Posts: 3 559 |
It would not be on each boot. If there is no data older than N days, or N = 0, it will simply start without doing anything.
Though, in your case it is probably deleting old data. Try opening the database in any SQLite manager and executing:
This should start everything over and allow you to open NetWorx.
Though, in your case it is probably deleting old data. Try opening the database in any SQLite manager and executing:
DELETE FROM appname; DELETE FROM sitename; DELETE FROM appusage; DELETE FROM siteusage; DELETE FROM appsiteusage;All usage data figures will stay, just without the apps and sites details. Then compress the database (VACUUM).
This should start everything over and allow you to open NetWorx.
Eco-Drive
Re: Delay in launch when the database contains a lot of data 28 February 2020, 15:37 |
Hi Andrew,
Thank you very much for the timely response!
Well on my computer, before I modified "KeepSiteInfoForDays", it would definitely take about 45 minutes for each program launch, during which I can see the size of "Networx.db-wal" slowly increases to the size of Networx.db, and disk usage locked to 100%.
Maybe there was already some corruption with my database, such as some key responsible for tracking the age of data not being set properly?
I wonder if the database already have some issue, causing networx to scan and compress the database on every launch when "KeepSiteInfoForDays = 0" and freeze without doing anything when "KeepSiteInfoForDays = 24"
Thanks.
Thank you very much for the timely response!
QuoteIt would not be on each boot. If there is no data older than N days, or N = 0, it will simply start without doing anything.
Well on my computer, before I modified "KeepSiteInfoForDays", it would definitely take about 45 minutes for each program launch, during which I can see the size of "Networx.db-wal" slowly increases to the size of Networx.db, and disk usage locked to 100%.
Maybe there was already some corruption with my database, such as some key responsible for tracking the age of data not being set properly?
After I modified the hidden option to 24 days, the next program launch it did not look like it was doing anything - in resource monitor the process shows zero disk activity. I waited for a few hours before killing the process in taskmgr.QuoteThough in your case it's probably deleting old data.
I wonder if the database already have some issue, causing networx to scan and compress the database on every launch when "KeepSiteInfoForDays = 0" and freeze without doing anything when "KeepSiteInfoForDays = 24"
Thanks.
Re: Delay in launch when the database contains a lot of data 28 February 2020, 16:45 |
Admin Registered: 18 years ago Posts: 3 559 |
Eco-Drive
Re: Delay in launch when the database contains a lot of data 02 March 2020, 02:16 |
Eco-Drive
Re: Delay in launch when the database contains a lot of data 02 March 2020, 13:54 |
Dear Networx developers,
Thanks for continuously monitoring the forum and provide support.
I want to add that the performance of "Keep app/site data for days" option needs improving. Currently, when this feature is enabled, program startup time is too slow even with a very small database.
For example, Networx will take about 3 minutes to start for a freshly created database with just half day of data (~7MB), which I think is beginning to fall into "unusable" category. A novice user might just assume the program is defective. Are there any possible remedies for this? Or is the performance of sqlite just not enough?
Thanks!
Thanks for continuously monitoring the forum and provide support.
I want to add that the performance of "Keep app/site data for days" option needs improving. Currently, when this feature is enabled, program startup time is too slow even with a very small database.
For example, Networx will take about 3 minutes to start for a freshly created database with just half day of data (~7MB), which I think is beginning to fall into "unusable" category. A novice user might just assume the program is defective. Are there any possible remedies for this? Or is the performance of sqlite just not enough?
Thanks!