Store phpBB3 attachments in subfolders
Posted April 12th 2015, 7:34am
Dion I just discovered your site and was doing some reading here I like your solutions I have read so far, have you ever given any thought into creating a mod that will put attachments into sub folders this is a modification I could really use as I have more than 47k attachments on my site.

Having more than 10k files in a single folder creates all kinds of problems when it comes to backing up and or moving the files. Not sure if you have ever tried to manage 47k+ files in one directory or not but its constant problems at every step of the process, in theory it should not be a problem until you try to do it then you discover the true limitations of the computer systems your working with server side and at home, its a stress test every step of the way, time outs, stalls, what should take an hr to accomplish sometimes turns into all day finding a solution.

Their have been attempts at this in the past but their never seems to be a polished finished project, seems soon as the author meets his personal needs the project stalls (hooray for me well good luck to you thing).

One solution I have been looking at is https://www.phpbb.com/community/viewtop ... &t=2130170 I tried it in the early days of development but ran into problems and getting support then was not good I put off the idea in hopes PHPBB would finish it and make it a new feature of PHPBB I get the impression developers at PHPBB could care less since they never experience the problem.

Since I first started using PHPBB back in 2003 I have always felt a great resistance to attachments when seeking support.

I think its time I finally address the 47k files in one folder issue since PHPBB is not before I waste a bunch of time contacting my host having them create a backup of my "Files" directory with the 47k+ files, downloading it verifying all the files are their then testing the mod on my home server http://www.uniformserver.com I figured I would ask here first.

Its obvious you have considerable skills where as I don't, I can follow direction and have a good attention to detail I just need the "for dummies" version lol

My needs are to put attachments into more than one folder each folder should have less than 10k files, I was thinking 9500 files per folder, I have tested with working with 9500 files in a single folder and while I found some slow downs I have not run into timeouts yet.

Sorting if not a big deal I would like them sorted chronologically so as time goes on newer files are put into newer directories, when one directory reaches 9500 files a new one is created then the files are put into the newest directory.

I don't have any concerns about older directories meaning once a directory fills to 9500 files then say 1000 files are deleted off the PHPBB site having a older folder with 8500 files in it. Chances of 1000 files being removed is unlikely.

Any thoughts on this I like your outside the box approach to things, I have found a very inside the box approach on the PHPBB site and it appears to be a very tightly sealed box so no thinking can happen outside when it comes to attachments.

Thanks in advance.
φ
Posts: 8
Joined: April 11th 2015, 8:31pm
Store phpBB3 attachments in subfolders
Posted April 12th 2015, 5:43pm
That particular MOD seems to be well-written and should be able to support what you want. I didn't check closely enough to see whether it supports existing attachment directories, however...

The sad thing is that all it would take to solve this problem is one additional column in the _attachments table for the directory where the attachment is stored, one additional _config variable for the number of attachements in the current working attachments directory, some code to determine where to store a new attachment, and some minor changes to the attachment display script (download/file.php) to utilize the new column in the _attachments table. FYI, I'd set the max number of files in a directory to 4000 because some servers don't like more than 4096 files in a directory.

I am currently working on a fork of phpBB 3.0.13 called php-bbdr. I have been thinking seriously about overhauling the archaic attachment "feature" in phpBB (it's archaic in phpBB 3.1 as well) and creating a media upload feature somewhat similar to what is in WordPress. It won't make it into the initial release, but I'm pretty confident it will be in the first update.
φ
Posts: 1599
Joined: March 12th 2009, 11:00pm
Location: Uncertain due to momentum
Likes Given: 26
Likes Received: 357
Store phpBB3 attachments in subfolders
Posted April 12th 2015, 8:58pm
Thanks for the reply I will give the mod I linked to a go then.

I was unaware of the 4096 limit I will go with 4000 files per directory thanks for the heads up.

I would like to see a demo of the php-bbdr when you get that far sounds like a interesting project if your going to overhaul the attachment thing how about where a user can do a batch upload so you don't have to go find a file each time for each attachment, it really sucks when you have 8-10 pictures you want to attach to a single post.

Thanks again.
φ
Posts: 8
Joined: April 11th 2015, 8:31pm
Store phpBB3 attachments in subfolders
Posted April 13th 2015, 2:15pm
Ok laying in bed last night thinking, would it be easier for me to convert my phpbb installation with another brand of forums that already is dealing with attachments by putting them into separate folders then using one of the phpbb converters convert my forums back to phpbb? Just a thought, currently trying to get my Uniserver back up and running so I am not testing on a live site.

Thanks.
φ
Posts: 8
Joined: April 11th 2015, 8:31pm
Store phpBB3 attachments in subfolders
Posted April 14th 2015, 12:52am
The one problem with that approach is that something always gets lost during conversions. However, given your situation it may be the best way to proceed. I don't know which types of board software handle attachments in the way you need, but from a conversion standpoint, SMF seems to lose the least during conversions.
φ
Posts: 1599
Joined: March 12th 2009, 11:00pm
Location: Uncertain due to momentum
Likes Given: 26
Likes Received: 357
Store phpBB3 attachments in subfolders
Posted April 14th 2015, 1:13am
Thanks for the info, after more thinking I will reserve this idea as a last resort.
φ
Posts: 8
Joined: April 11th 2015, 8:31pm

Who is online

Users browsing this forum: No registered users and 1 guest