FlashFXP Forums - View Single Post

Yil · 04-12-2010, 04:18 PM

Here's a few thoughts on the issues...

At one point I had some code in the server to detect when disk space got below some threshold before creating a directory and it would then call a new OnDiskSpace event which would have the task of making space or aborting. I thought this was a simple solution, but decided against it. What really needs to happen (and will in a later release) is that the already called OnNewDir event should be able to choose where to create the directory as well as be responsible for making sure enough space exists or it should abort the creation. The "where" is of course limited to the various possible paths of the merged directory unless a pure virtual directory in which case it could be anywhere. EVEN WITHOUT THIS CHANGE you can already make space on demand in the OnNewDir event right now...

That is the easy part, the hard part is deciding what should be deleted to make room when there is no other choice. I freed up 3 bits of every directory permission (the +x bit of owner, group, and other) and suggested using one of these bits was my long term plan. This would allow you to make individual directory trees or single directories immune from wiping. There are many other ways to get the same result using a database, chattr directory attributes, .ioFTPD.file hidden files, etc. I just liked this idea because you could immediately see in the directory listing which dirs were exempt.

In my previous effort I had some glob patterns you could define in the .ini file to detect complete/incomplete dirs. There were also some simple rules like: any dir with no subdirs or only empty subdirs was a "release", and another set of defined glob patterns were applied to all non-empty subdirs in a directory and if all matched then the directory was considered a "release". That's pretty common and simple stuff used by many scripts. Then the ages of directories could be sorted and the oldest non-exempt directory could be removed. Compiling all that info on demand all the time would be really bad, so it really needs to be stored in a database which is why ioFTPD itself will never auto-wipe stuff but must rely on a script to do it.

It's immediately obvious however that using that simple set of rules above could lead to some real problems. Rescanning/updating of imdb links in an XVID directory might adjust the directory timestamps in such a way that new stuff got deleted first. So it would be important to use the creation time rather than the last modified disk timestamp, but internally ioFTPD doesn't care or track creation time. A simple query against something like the nxTools dupedb which is based on creation time to get the oldest non-exempt directories on a particular disk would be the simplest solution. But what if you wanted to keep some directory trees longer than others but they are on the same disk? That's the specification problem and it's tricky. I don't presume to know the best solution, but that's why having a script do it makes sense since it would be easier to customize.

The most important point here is that everything but spanning disks on directory creation can be done today by just adding the functionality into something like nxTools. There really isn't any reason why the simple solution outlined above wouldn't solve the problem for 90% of users...

04-12-2010, 04:18 PM
Yil Too much time... Join Date: May 2005 Posts: 1,194	Here's a few thoughts on the issues... At one point I had some code in the server to detect when disk space got below some threshold before creating a directory and it would then call a new OnDiskSpace event which would have the task of making space or aborting. I thought this was a simple solution, but decided against it. What really needs to happen (and will in a later release) is that the already called OnNewDir event should be able to choose where to create the directory as well as be responsible for making sure enough space exists or it should abort the creation. The "where" is of course limited to the various possible paths of the merged directory unless a pure virtual directory in which case it could be anywhere. EVEN WITHOUT THIS CHANGE you can already make space on demand in the OnNewDir event right now... That is the easy part, the hard part is deciding what should be deleted to make room when there is no other choice. I freed up 3 bits of every directory permission (the +x bit of owner, group, and other) and suggested using one of these bits was my long term plan. This would allow you to make individual directory trees or single directories immune from wiping. There are many other ways to get the same result using a database, chattr directory attributes, .ioFTPD.file hidden files, etc. I just liked this idea because you could immediately see in the directory listing which dirs were exempt. In my previous effort I had some glob patterns you could define in the .ini file to detect complete/incomplete dirs. There were also some simple rules like: any dir with no subdirs or only empty subdirs was a "release", and another set of defined glob patterns were applied to all non-empty subdirs in a directory and if all matched then the directory was considered a "release". That's pretty common and simple stuff used by many scripts. Then the ages of directories could be sorted and the oldest non-exempt directory could be removed. Compiling all that info on demand all the time would be really bad, so it really needs to be stored in a database which is why ioFTPD itself will never auto-wipe stuff but must rely on a script to do it. It's immediately obvious however that using that simple set of rules above could lead to some real problems. Rescanning/updating of imdb links in an XVID directory might adjust the directory timestamps in such a way that new stuff got deleted first. So it would be important to use the creation time rather than the last modified disk timestamp, but internally ioFTPD doesn't care or track creation time. A simple query against something like the nxTools dupedb which is based on creation time to get the oldest non-exempt directories on a particular disk would be the simplest solution. But what if you wanted to keep some directory trees longer than others but they are on the same disk? That's the specification problem and it's tricky. I don't presume to know the best solution, but that's why having a script do it makes sense since it would be easier to customize. The most important point here is that everything but spanning disks on directory creation can be done today by just adding the functionality into something like nxTools. There really isn't any reason why the simple solution outlined above wouldn't solve the problem for 90% of users...