3. Seems best if one person runs the tool and collects the data, as opposed to everybody (loosely used) running it individually and hitting Jeff/George Soros with a big bandwidth bill. Ideally, that person would review the files for issues like those above, and post samples for knowledgeable members to assist with remediation. Once complete, that person could zip up the whole thing and post it to a file-sharing site.
That sounds like the right idea. Especially since an equivalent to Forum Downloader doesn't exist for osx. There are site-suckers, of course, but can't find one specifically designed for forums.
Just ideas and several other possibilities are imaginable. As it stands, anybody using Windows can download the software now and pull down a nearly perfect version of the forum. I wouldn't be surprised if this tool produces a more user-friendly end-product than even what somebody with admin access could provide. Voilà, wookies.
Thanks for setting it in motion.
Re: Engagement, Attention, and Death
Posted: Thu Oct 27, 2016 4:45 am
by Joao
After spending more time with the software, the process has proven more difficult than expected.
Thread and forum pagination aren't detected properly by the automated algorithms, which leads to significant gaps in the downloaded archive. User details associated with each post are also captured erratically.
Manually configuring the "detection" screen has promise, however, and some headway was made with the pagination issue. It would take careful review of the forum's HTML source to get all the specifics right.
I'm also encountering an issue where the software is misreading forum links, resulting in File Not Found errors. Again, configuring the "detection" screen properly may help.
Perhaps someone else has the patience and interest to pick this up. This is still the best tool I can find for this purpose. Searching the web reveals a lot people trying to do the same thing but very few helpful answers. I did see some purported solutions which rely on Ruby or Python, but those are over my head.
Re: Engagement, Attention, and Death
Posted: Thu Oct 27, 2016 6:08 am
by tron
how is jeff doing? does he still pay the bill for this place? maybe someone who deals with the tech side of the forum could ftp dump it and share it in one big chunk?
Re: Engagement, Attention, and Death
Posted: Thu Oct 27, 2016 9:37 am
by Luther Blissett
We raised funds a few years ago, maybe we exceeded. Might be time to kick again.
Yeah Ruby or Python would do it. I'm sure there's some script kid on youtube with a tutorial.
Yeah Ruby or Python would do it. I'm sure there's some script kid on youtube with a tutorial.
In the Python world there's Scrapy, which is a framework for building a task-specific web scraper. This guy posted one for phpBB a year ago (haven't tried it):
1 night while sleeping in my bed
eye had a beautiful dream
all the people of the world got together
on the same wavelength & began helping one another
now in this dream
universal love was the theme of the day
peace & understanding
it happened this way
Wombaticus Rex » Tue Oct 18, 2016 7:01 pm wrote:As the cycle of Mundane & Meaningless accelerates, I do wonder what RI might return to on November 12th (after the riots die down).
