RAM disk .IMG file corruption

Started by gldisk

RAM disk .IMG file corruption   29 July 2021, 03:38

I have tested SoftPerfect RAM Disk 4.2.0 demo on different computers but with identical configuration, so that it saves its contents in a 1 GB .img file on a separate non-boot drive with the options "Save contents to image" and "Save contents to associated image every 60 minutes". The RAM disk contains the Firefox browser profile and is formatted with the NTFS filesystem on Windows 10 Enterprise Ltsc operating system, configured as a boot disk with "Hard disk emulation" option unchecked.

It's the second time, on a different PC, that after restarting Windows10 and accessing the RAM disk I get a message that says that the disk is not formatted. From the SoftPerfect RAM Disk interface, the disk still appeared correctly mounted, but there was no way to access its contents. I've recovered the data within the .img file using a data recovery program, but almost all files were corrupted.

I'm concerned that the image corruption may be caused by the automatic saving of the disk image being interrupted by the OS shutdown.

Compared to a backup of the .img file, the corrupted one appears increased in size by 4 KB, exactly like the first time I experienced this problem.

The antivirus is the Windows 10 default (Windows Defender) and the folder where the .img file is saved is excluded from its scan.

I have seen that the same problem has been reported by other users. Is there any news about the solution of this issue?
SoftPerfect Support forum - Andrew avatar image

Re: RAM disk .IMG file corruption   29 July 2021, 10:09

Unfortunately it is still unknown why this happens, so we recommend having a frequent backup of any image files. The vast majority of users never have any issues, but there are clearly some who experience this problem. Some users mentioned that they had PC clean-up programs like CCleaner, and those somehow interfered with the image files and caused corruption. Unfortunately we were unable to reproduce this in our tests.

By the way, a file recovery program may not be able to deal with these IMG files correctly, and that is why you see all your files as currupted. Despite having the same extension, the format is different from regular disk images. There are two identical 4KB headers: a primary header in the beginning of the file, and a backup header in the end of the file.

To open the IMG file try a third-party disk mount utility that supports entering an offset, like OSFMount v2 (see under Downloads/Previous version). You need to enter the offset of 4096 bytes to the data when mounting the image file, as shown below. An image file is essentially a 4K header followed by raw disk data.

Regarding your specific usage, i.e. having a Firefox profile on a RAM disk, it may be better to have the profile on an SSD. In that case you could move only the Firefox cache to a volatile RAM disk (without image file). All cache will be cleared on restart and recreated next time you use Firefox. There would be no data loss as the cache does not need to be persistent.

SoftPerfect support forum

Re: RAM disk .IMG file corruption   30 July 2021, 06:43

Hello and thank you for your reply.

The Windows 10 configuration has remained pretty much the standard one and no programs like CCleaner or similar have ever been installed. Unfortunately, also, the suggested method of accessing the corrupted image file by mounting it with OSFMount v2 does not work.

I suppose that the corruption problem is not with the data in the ramdisk, but occurs when writing this data to the disk image. This hypothesis is supported by the fact that there are no malfunctions while using Firefox, while the described problem always arised at start-up, when the disk image was loaded again.

If possible, I suggest to add a function so that the image file used is not just one, but at least two of them that are written alternatively. In this way, when the described problem occurs, the corruption should affect only one of the two files.

Perhaps, enabling also a debug function by volunteer users would also help to understand the cause of the problem.
SoftPerfect Support forum - Andrew avatar image

Re: RAM disk .IMG file corruption   30 July 2021, 10:42

It's a little complicated with writing into two image files. We use partial writing where an image is divided into 2 MB blocks and only changed blocks are written back to the image. For example, if you had a 1 GB image and only a couple of small files were changed, all we need to write is 2-4 MB of the changed blocks rather than the entire image. This greatly reduces SSD's wear-and-tear and increases performance. Writing into two alternating images would therefore require to overwrite the entire file, which is slower and puts a strain on your SSD.

Ideally, in order to be able to figure out why this happens we would need two images, i.e. uncorrupted and larger corrupted one, so we can do a binary compare and see what was written and where. Since it doesn't really happen in tests, would you be able to create another image-based RAM disk and use it for some non-sensitive data (e.g. temporary files)? If so, and the corruption takes place, please send us the corrupted image and the uncorrupted image. You could simply make a copy each time it's successfully mounted and until it breaks.

Other than that, we will also try running a longer test with a similar configuration to see if we can reproduce this issue.

Re: RAM disk .IMG file corruption   02 August 2021, 07:05

Quote

Andrew

It's a little complicated with writing into two image files. We use partial writing where an image is divided into 2 MB blocks and only changed blocks are written back to the image. For example, if you had a 1 GB image and only a couple of small files were changed, all we need to write is 2-4 MB of the changed blocks rather than the entire image. This greatly reduces SSD's wear-and-tear and increases performance. Writing into two alternating images would therefore require to overwrite the entire file, which is slower and puts a strain on your SSD.


Why? In case of modifications to blocks 4,5,6 (just to make an example) only these will be written to the first image. The next time you save on the second image, blocks 4,5,6 will be written to it in addition to only the other modified blocks. But, maybe I'm wrong.

Quote

Andrew

Ideally, in order to be able to figure out why this happens we would need two images, i.e. uncorrupted and larger corrupted one, so we can do a binary compare and see what was written and where. Since it doesn't really happen in tests, would you be able to create another image-based RAM disk and use it for some non-sensitive data (e.g. temporary files)? If so, and the corruption takes place, please send us the corrupted image and the uncorrupted image. You could simply make a copy each time it's successfully mounted and until it breaks.

Other than that, we will also try running a longer test with a similar configuration to see if we can reproduce this issue.


Is it possible to log successfully save operations, to see if the problem is caused by the image writing process being interrupted when the operating system is shut down?
SoftPerfect Support forum - Andrew avatar image

Re: RAM disk .IMG file corruption   02 August 2021, 12:39

Unfortunately it's not that simple. RAM Disk would need to keep a list of blocks somewhere else in an external file, deal with keeping two images up to date, and that's a lot of extra functionality and instability risks for something that is an auxiliary function and something that may not even solve the original problem.

On a related note, we did run a continuous cycle of writing random files to an image-based RAM disk and rebooting at random time intervals. The disk was 4 GB image-based disk set to update the image once a minute. The cycle went over ~50 reboots over a few hours and the image was successfully updated even when timed writing coincided with system shutdown.

Considering it did not break in stress-testing, it is safe to say this image corruption is very infrequent, and the cause remains unknown. It may well be a bug in a different application installed on the same computer, or a hardware issue, or a rare coincidence of factors, other than the writing during shutdown. If it is essential to you, I would recommend trying a different product or doing a backup each time the system boots.

Reply to this topic

Sometimes you can find a solution faster if you try the forum search, have a look at the knowledge base, or check the software user manual to see if your question has already been answered.

Our forum rules are simple:

  • Be polite.
  • Do not spam.
  • Write in English. If possible, check your spelling and grammar.

Author:

Email:

Subject

A brief and informative title for your message, approximately 4–8 words:

     

Spam prevention: please enter the following code in the input field below.

 **     **  **     **  **      **        **  **      ** 
 ***   ***  **     **  **  **  **        **  **  **  ** 
 **** ****  **     **  **  **  **        **  **  **  ** 
 ** *** **  *********  **  **  **        **  **  **  ** 
 **     **  **     **  **  **  **  **    **  **  **  ** 
 **     **  **     **  **  **  **  **    **  **  **  ** 
 **     **  **     **   ***  ***    ******    ***  ***  

Message: