Well, it has been almost two months since the technical support people at my website hosting partner accidentally nuked my site. The changes they made screwed up the site and the backups wouldn’t restore properly, leaving me with a huge reliability problem for some of the content. Pages that would work one minute and crash the next, posts that wouldn’t display properly, some pages or posts that disappeared altogether yet were still “active” on the site. Glitches that were annoying and not something I could leave in the site nor was I able to simply “fix” them.
I had all my data, but I needed to nuke the account and rebuild everything entirely. I rebuilt two sites that I host for others, and the content of those was relatively easy with a few annoying features, but it was the main PolyWogg.ca site that was the biggest challenge. 200 pages, 1600 posts, 1.6M words, 400 comments…all needing to be reloaded. And, as I noted, some glitches were embedded in several posts that required me to essentially reload things manually.
Working around the clock
But after two weeks of existential angst and six weeks of working at least 4h/day (at least 10h/day on weekends), and about 260 hours of work in total, my sites are back in business.
Because I had to rebuild the sites, and I didn’t want people getting daily update notices, I had to “force unsubscribe” anyone who was subscribed to my original feed for PolyWogg. Back on March 6, I sent them all emails to say, “Well, the site is down, and I’ll email you again when it comes back up”, while redirecting people to Facebook or Twitter to find me for now or in the future. I’m a little sad that many of them won’t resubscribe, but that’s the nature of the beast when running websites.
Today, I re-invited everyone to rejoin, we’ll see how many do. I now have two feeds though, one for each website.
PolyWogg.ca which includes my more formal “products”, such as my HR Guide, reviews, recipes, and astronomy-related posts;
and
ThePolyBlog.ca which includes my more informal blog posts, such as posts on Life (experiences, family, humour and computer-related topics like the website itself); The Little Grey Cells (more thoughtful topics like learning and ideas, writing and publishing, quotes, and health and spiritualism); goals; and my annual reading challenges.
If anyone else wants to subscribe by email, here are the two links:
Of course, people may prefer to access my posts through Facebook (PolyWogg.ca) or via my Twitter feed (@ThePolyBlog), as I try to post notices of new content to both feeds. I had to “force stop” the feeds during the rebuild process too, albeit one seemed to occasionally update anyways on Facebook for a few posts, but everything is back live again.
Of course, the work is never done
I have a few areas that will require more work still, but that isn’t entirely due to the website torching. As I rebuilt, I took the opportunity to redirect some of my efforts in a bit different direction. As a result, some of the solutions are not totally compatible with the previous structure. My trivia efforts had to be reset, my photo gallery is moved over to Flickr, and I’m taking a whole new approach to my PolyWogg Guide to Astronomy. Most of that change though is that as I did the other changes, I figured out better ways to handle those options too. That’s a longer term project though, and I’ll get to it over time. I don’t need to relaunch with those today.
On the other hand, I’ve done pretty well. Both sites (PolyWogg.ca and ThePolyBlog) are running about 45 plugins in total, and about 20 or so are new ways to handle the content (replacing older plugins that I was running before). I’ll blog about that in a bit more detail as the week goes on. More importantly to the reader, I have 604 posts / 45 pages / 570K words on PolyWogg.ca. Plus about 230 comments, most related to my HR Guide. Meanwhile, ThePolyBlog has 879 posts / 14 active pages / 981K words, plus about 220 comments. I’ve also streamlined the categories on both sites quite a bit.
As the planet starts to emerge from the COVID pandemic, one of the phrases being used is “Build Back Better”. I’m not 100% certain that it’s perfectly re-built, but it’s definitely better than it was, with more extensive branding for sub-areas, particularly the PolyWogg “products” like my HR Guide.
Over the last few years, I have been increasingly active in a few active fora online. In particular, multiple ones related to amateur astronomy and a couple related to WordPress. My participation is generally two-fold — I both learn from others and help newbies with their questions. I’ve also been active in the past in groups related to my son’s health issues, although those are often more sharing of experiences than directly helping anyone.
One thing that I am particularly good at is framing information in a way that a newbie can understand it, orient themselves to the topic, and then proceed with a rudimentary schema for processing various types of other information coming their way.
Yet what I am often struck by, and amazed at, is the completely useless advice provided by well-meaning people in the fora. I don’t mean that the information is wrong, although that happens too. I mean people who ask a question, and then when someone responds, the response is of the form “Do x.”
At first blush, that sounds helpful, doesn’t it? They asked what they should do, and somebody answered. Except they didn’t ask any questions to understand the context for the question. They didn’t say to the newbie, “Wait, what are you trying to do first? What are you looking for?”.
In astronomy circles, there is a very common opening salvo by newbies. “Hi, I’m new to the group but I’ve always been interested in astronomy. I have read a bunch of stuff online, but I’m just more confused. I want to buy a scope, and I don’t know which ones are good for newbies. Please advise.”
And invariably, within about 3 replies, someone says, “Buy a Dobsonian” and someone else says “buy binoculars”. Is that wrong advice? Maybe not.
But it’s about the equivalent of someone asking what tool to use to take apart a workbench, and someone saying a saw, another saying a screwdriver, and another saying a hammer. Are they wrong? Not really. But until we know the context of how that workbench was initially assembled, what it’s made of, and what the person intends to do with the materials afterwards, the answers are at best incomplete. As a result, they’re useless.
For astronomy, for example, telling someone to buy a Dobsonian design is about the same as telling someone to buy a sedan when looking for a car. It’s an all-round good choice, good value, a solid utility vehicle. On the other hand, if the person was looking to haul equipment around their farm, not the best of choices. Yet people will tell someone to buy a Dob without ever asking a single question about what the person is looking for, are they comfortable learning to navigate the sky manually, are they looking to get into astrophotography at some point, etc. Equally, binos are a frequent “all-round” suggestion EXCEPT it assumes that the person is able to stand still (i.e., no wobbles, no physical mobility issues) and their eyes work well in conjunction with each other and don’t have any aggressive astigmatisms or wear bifocals. If either of those is not true — i.e. for kids who can’t hold heavy binos still or seniors with different eyesight profiles — then binos might be a terrible suggestion. They are also less useful for certain types of objects (moon, planets) which are quite often VERY popular starter targets. In addition, they require manual navigation of the sky too, which might not be what the person prefers.
In WordPress, people frequently say “Get Elementor” which is a page designer. It is advertised as easy to learn, easy to use, and there is common wisdom out there to regularly say “it’s free and has lots of power”. I have been using WP for close to 10 years, and when I tried Elementor? I found it completely confusing. Plus it mucked with a bunch of my existing setup. It was an overly complicated and terrible design for a newbie who doesn’t even know what WP does or a theme is for, let alone plugins, but the common advice is to start with full page design to start?
I don’t know what it is, whether it is group think, or the dangers of underestimating stupidity in large groups, but frequently I will see people chiming in and leading the person down roads they are likely to follow, get confused, get frustrated, and end their trip before it even begins.
Equally in astronomy, there are certain types of setups that are better for astrophotography than others. While you can do visual and AP with them, they’re often not as versatile or as simple for visual. Different tools for different projects, so to speak. Yet there are people who say “Buy this equatorial mount” which is good for AP more than visual and is priced at 5x the budget the person said they had, or add this gadget, buy this upgrade, get this accessory. It is very popular for some people to randomly suggest upgrades to gear when it is someone else’s money.
Most days it’s merely puzzling. Some days it’s maddening. I’ve sometimes come to a discussion late, 40 people have already taken the person down ten different rabbitholes, I ask two simple Qs, and it turns out NONE of what the guy already went through applies to his situation. But nobody asked what the problem was, they just started throwing out generic solutions.
I feel at times it’s like the old issue of someone typing “FIRST!” when commenting on a post. Stupid and pointless. But then I feel more angry later…because in a couple of cases, the answers were SO wrong, that the person felt like they had NO choice but to give up the hobby. Because they had a budget for astronomy, for example, of $200, and some moron told them they needed to spend at least $1500 or it wasn’t even worth it to get started.
I confess that more and more, this kind of misleading “advice” just pisses me off. The person has no way of knowing the advisor is well-meaning but stupid. I’ve seen it with people trying to understand French training, HR prep for exams and interviews, or writing fiction too.
Maybe I’m cranky, but I feel like if you don’t have something actually useful to contribute, then don’t bother saying anything at all. 🙂 Others are more of the “you get what you pay for” variety. Either way, I am starting to believe in the “superficiality of the crowd” more than in their wisdom.
I like running my blog, but I confess, I’m not a big fan of the upkeep behind the scenes. Basic stuff is fine, it goes with the territory. I could pay someone to maintain my site, but since my site does not and will not ever generate revenue, I need to keep the costs as low as I can.
But some of the overhead is of my own making, sure. Or perhaps even a result of my own ambition. I’m not satisfied with using some site like Amazon Photos or Flickr, I want my own site to host my pics.
And so I have some extra admin headaches managing a large collection of photos and galleries. Like the fact that I noticed that some of my Galleries had inconsistent structures to the weburl. What does that mean? It means that the right setup is supposed to be, say for my seventh gallery of my wedding collection from 2008:
URL: http://www.polywogg.ca/pandagallery/yr2008/w07-bachelor party
Page title: 2008-W07 Bachelor Party
Gallery: 2008-W07 Bachelor Party
The URL had yr2008/2008-w07 (2008 listed twice), the page title was fine, and the gallery in some cases was messed up and said 2008-2g Bachelor Party because that was the old filenaming I did, and it hardcoded it into the directory structures. Is it a problem?
Yes and no. It starts off no but quickly becomes a yes when I start linking to things, and suddenly something that should link just fine, instead has a really weird file structure to it and a picture doesn’t load when it should. I was ducking and covering, letting it go, until I hit a snag tonight reading where the WordPress environment is going and realizing my little booboos are going to become bigger booboos later and it is better to fix them now.
Okay, it’s annoying, but an hour or two of quality control has me having corrected some past mistakes and I’m good to go again. Except I’m not.
I’ve got a gremlin. I don’t know what’s causing it, I don’t know when it started, I just know that for some reason, a gremlin is visiting my website.
Here’s the deal. For my gallery, I basically say “CREATE NEW GALLERY”, type in a gallery name, press enter, and then drag and drop all my photos for that gallery into the browser window. I press UPLOAD, and it puts everything onto the server and into the gallery for me. Easy peasy lemon squeasy.
It has worked this way for years. And now it doesn’t.
For some reason, my website has decided to time out in the middle of file uploads. Oh, no, not consistently. Just enough to be a PITA. It gives me an error message, saying out of 21 photos only 16 uploaded. So I note the five that it says didn’t upload. I reupload those. And it rejects one as already uploaded. Hmm…Okay, whatever, it took the four. So I go to the gallery and it says I have 24 photos. Wait, what? I had 16, I added 4 new ones, I should have 20 perhaps or 21 if the duplicate was an error. But how did I get 3 more than I started with?
Oh, right. Not all of those errors were actual errors. In fact, of the 5 it said it didn’t upload, it actually uploaded 4 of them. So I was only missing 1, right? Nope. Another two didn’t upload. No error, should have worked, but when I look at the photo, it is blank or incomplete. Umm, okay.
Plus, for some weird and wonderful world, one of the pics that didn’t upload, and isn’t in the directory, did upload the second time as it should have, but for some reason and some how, WP decided it was a partial duplicate. Not enough to block it, but enough of a duplicate to rename my filename to something like 1_image_xxxx.jpg.
Son of a firetruck.
Deep breath, dive into the rabbit hole.
Generally speaking, a problem with uploading pictures is USUALLY a problem with certain settings. And there are sites that list what the most common problems are for that error, and 8 different ways to fix them. Except none of those 8 ways apply to me. I know, because I had another problem back in December that WAS of that type and I fixed it. Or so I thought.
The app that is giving me errors is one I actually paid for, including support, so I’ve reached out to them for official suggestions. They gave me two solutions, I tried both, seemed okay, thought I was sorted, but tonight, it started throwing errors again.
Okay, debug time. What is the first step? Deactivate a bunch of other plugins to see if they’re conflicting. Yes, it works without them. Great, slowly reactivate them one by one until you find the one that’s a problem. It isn’t usually necessary to literally go 1×1, I find 5×5 is a good enough sort technique, reminiscent of advanced sorts in programming. I got about halfway through my list of plugins and hit a conflict. It stopped working. Great, that’s the problem.
Oh wait, I put another 5 in with no problem and then the next 5 conflicted again. Wait, there’s more than one conflicting? That’s unusual. Particularly for the narrow area I’m having the error in. Most plugins would never conflict with that. Weird. Okay two conflicts.
No, wait, a third. And a fourth? WTF? A fifth and a sixth? That is NOT possible. The plugin works normally, it’s a simple upload feature. It’s used by hundreds of THOUSANDS of people around the world. I know it works. WTF????
Okay, so it isn’t a real conflict. Maybe it’s a load issue? Like if I load 20 plugins, the server doesn’t want to run 20 anymore? I have 40 in total, mostly light load. They should run fine.
Sigh. Okay, I’m out of options. I reached out to the plugin again to see if they have other suggestions, and to my official server support to see if they know what would be causing it to time out after 30s when every variable in the system is at least 60s and I’m not over any obvious memory load. Hmm…
Why is this a big problem? Because it means I can’t easily upload my pictures right now.
Don’t get me wrong, there are other ways to put them on the server, including the most obvious one is to do a side-load equivalent to upload them using better software directly to the server and then have the app “copy” them from the server into the gallery. It adds about four steps to my workload to do all that, and it’s a totally separate PITA.
I can do it, but I sure don’t want to. I need to fix the original problem.
The plugin support did mention they have a beta app coming soon which would potentially indirectly fix my problem, it would just potentially slow the upload to a crawl. I CAN do that, I don’t know if I WANT to do that.
Did I mention that some days I think it’s a dumb website?
If you have archivist tendencies, combined with strong analytical props, and a digital bent, digital photo management is the field for you, my friend. As an amateur or professional, you too can find new and interesting rabbit holes to explore.
For me, I have wanted to put some of my photos online since 2005 when we bought our first digital camera. Actually, a little before that, as we had films developed and they threw in digital prints for a $1 more sometimes. And I’ve had a website since back to the dark ages before that, with the natural thought, “Could I share them on there?”.
Round 1 – Basic HTML site
Initially, round 1 of my attempt was to code my own little gallery website. I was doing all of my photos in custom FrontPage designs, and I uploaded two or three albums in HTML code. It worked, it was functional, but it wasn’t very easy to manage. More like “dump them here and you can see them”. I also wasn’t particularly sure if I had a good process behind the scenes, but when there’s only a couple of hundred, the supply side of uniformity in file management gives way to the demand side of ease of management. Throw them in a folder, call it done. Upload a few, call it done.
Eventually, I upgraded the side to a content management system, and photos were NOT an easy inclusion. Sure, I *could* include them, but it was very manual AND hard to manage all the photos on the site. They didn’t go in a subfolder, they just went in one big directory online called “media” or “images”. Not very satisfactory even with a couple of hundred images. If I uploaded another year’s worth? Meh.
Sure, I could do it through FB, but I don’t really like FB that much, and I really don’t like having all my stuff there. Plus lots of people in the family who would/could/might want to view the images aren’t ON Facebook. Or at least most of them weren’t at the time. Over time, that edge has dulled a bit, but still, it’s an issue.
Round 2 – Photo hosting site
Eventually, I decided I needed a REAL solution for online. I had enough photos that I wanted to upload that I went for an online photo site, and reviewed a bunch of sites. In the end, I went all-in on Smug Mug. It was great. I could choose a theme, I could have subdirectories. I had to manually add all my labels and descriptions, uploading my initial pics for the second time plus about 4y worth of photos, but at least I could do it. Cross-linking to my website wasn’t easy peasy, lemon squeezy, but I could do it. Sort of. More like “good enough” rather than “good”.
Except that because of the volume and use I was needing, I needed a paid account. It was only about $100 a year overall, but it was always a bit grudging payment. Here I was, paying for Smug Mug to host my pics, when I was ALREADY paying for my own site. Does that make sense to anyone? Easier than running my own photo gallery, most of the time, sure, but on principle, it annoyed me.
Round 3.1 – PolyWogg.ca
And about 6-7 years ago, the principle got to me. I wanted it on my own site. No content rules, no limitations, my own site. So I moved it to my polywogg.ca account. Great. I ran a gallery called Coppermine initially, got it going, wasn’t totally happy with it, but managed to upload a year or two. Not bad. I considered it “round 3” for online.
But it wasn’t working quite the way I wanted it to, I struggled here and there. Eventually, I decided I needed a different solution, and opted for a photo gallery called Piwigo. It had a lot of power, extra extensions, themes and plugins, like any good online community eco-system. And it handled all of my photos REALLY well. Video was still a bit of a challenge, but I could make it work. Probably.
I didn’t quite get the chance to find out. I ran into some problems about that time with an old hoster, and moved to my third hoster of my online career. I lasted about 18m with that one before they really started screwing me around. I was almost to the point where I was considering calling in a lawyer if the amounts weren’t so small. Mostly I just wanted to smack them around. Really terrible business practices and even worse support. Like them modifying my site without telling me, my finding their changes, their denying it despite the logs showing they had done it, so they deleted the logs, and my support tickets, and then deleting my complaint files (all the same support people) so the bosses wouldn’t find out what they had done. Eventually it blew up on them entirely, and a lot of people went public for awhile before the whole unit was fired and supposedly new people hired, but by that time, I was long gone.
I had moved to a medium-sized company in Canada, my current hoster, and within days of moving, I knew I felt at home. There had been a long, lingering problem on the old site, I was convinced it was a server configuration issue but had no idea how to solve it and the support people denied there were any issues. Two days after I moved to the new host, their support group reached out to me, noting the misconfiguration was likely affecting my site performance and suggesting a fix, if I was okay with it. For what I was doing, there was a small reconfiguration required, and they were proactively helping me solve it. Nice.
I reinstalled Piwigo, spent about a year getting it all up and running the way I wanted it to (after all the other changes I made to my main website were taken care of), and I consider that round 3.2. I started uploading photos. Again.
For the first year of photos, this was the fifth time uploading them somewhere (once in HTML, once in SmugMug, once in Coppermine, and now twice in Piwigo on two different hosters). Some metadata transferred, some did not. Sigh.
Round 4.1 – WordPress
I know I’m anal, but this decision really wasn’t mine. Not exactly.
You see, my site has unlimited storage and unlimited bandwidth (within the general setup of the site for speed and servers), BUT one thing that almost all small hosting packages have in common is a small note in the fine print. A limit on “inodes”.
If you don’t know what an inode is, you’re not alone, and most people who have hosting packages never even notice it. It’s basically a “file marker” in the server that tells it where to find a folder or a file. Like an index card system in a library or your old file allocation table in Windows.
For my hosting package, I am authorized up to 200K inodes. Which sounds like a lot. I have unlimited space, but for inodes, I can have say 1 folder with 199,999 files in it, or 100K folders with only 1 file in it. Neither are likely scenarios but here’s the catch. When you install WordPress, with all its little files for the core, themes and plugins, it takes about 10K files and folders. Piwigo takes about 5K all on its own. I also run two other installations of WP on my site (for other sites), and I used to have 3. Which meant just based on “installed” software, I had 35K worth of my 200K inodes already taken up.
Still, lots of room, and I cut one installation when I merged PolyBlog with PolyWogg. Back to 25K in inodes, 175K left. Plenty of space, right? Except Piwigo has a really nasty habit of generating other sized photos. So let’s say I upload 10K photos. That’s 10K inodes. Initially.
Then Piwigo generates a thumbnal (+10K), a small image (+10K), a medium image (+10K), a large image (+10K), and the original image (~0 extra). So 10K worth of images generates 50K in inodes. Umm…that’s not good. In fact, with EVERYTHING running at one point, I was up to 145K/200K used. Yikes.
Now, I can reconfigure Piwigo not to do that, and I did. I got it down to a smaller number, but the way it does it, it will always generally be twice the number of inodes.
Okay, so I had it down to a smaller functioning site, all good, right?
Well, not exactly. I still had to keep maintaining the site for admin, including improved security, etc. Plus, it isn’t exactly the most robust of software packages. I found a few things that had to be coded manually to fix, and while we found solutions (or rather the community experts helped me figure it out), it was kind of like hacking the code to make it do what I wanted. Satisfying and unsatisfying at the same time.
Round 4.2 – New WordPress versions
In the meantime, WordPress was continually evolving. It moved forward several iterations and then finally a full version upgrade, and more and more, the Piwigo solution wasn’t really integrating very well, Which is a bit of a problem.
I am, primarily, a blogger. While I have a huge site, most of the content is in pages I wrote as blog entries like this one. And I want to include more photos. Even if it is only, “Hey, here’s this photo I took yesterday at the tulip festival” before I tell some story about the experience. Yet the more WP evolved, the harder it was to integrate the photos from the site. I did it a bit manually for awhile too, passing up on some malfunctioning automated tools, but it was far from satisfying.
What I REALLY wanted was what I had wanted from the beginning. One site, one solution.
I dug back into all the photo galleries that had existed from the dawn of time, or at least it seemed like it. I found dozens that were popular and in heavy use. Some were really cool. I limited myself to those that were still compatible with the new versions of WordPress, but it was still a long list.
And almost all of them had a recurring problem. The same one I had way back at the beginning…they all use the media library as their default save location, which means by default, all the photos are stored in the same place as where you store your site header, featured images, etc. It’s nice that it’s all in one spot, but it is kind of like throwing all your books in one room and saying you have a library. No organization, no easy searching, just a long list of images to find the one you want.
The most popular one of all is one called NextGen. It has been around for years, made by Imagely, and one of the reasons people use it, other than robustness, is that it has a totally separate file structure. That presents good and bad features, but the biggest “pro” is that all of your media is stored separately. Your core media library remains untouched. One “con” is that it doesn’t handle video.
But since none of the others can handle video either, I gave it a go. Again. Sure, I say again, because I had tried it 2-3 other times previously. I always wanted all my stuff in WP, and every time I considred Smug Mug or Piwigo, I looked to see lightly if I could find a good solution in WP, and NextGen was always on the list. I could never get it to work properly.
I don’t know exactly why, but it would NEVER work right. So I’d move on, frustrated.
This time I tried it, and it worked. Out of the box, day one, first light. It just worked. What’s different from the last time? A new version of WordPress, which is significant. And I’m on an entirely different hoster that is properly configured. Does that make the difference? I don’t know. I just know it works.
Holy crap. It worked. I could integrate my GALLERY within my MAIN SITE. Holy snicker doodles.
I started uploading. I got 2005-2008 uploaded, and I hit a small wall. My site design wasn’t quite right.
Round 5 – PolyWogg 5.0
I redesigned major parts of my site in the last year. Fixed a bunch of inconsistencies, tweaked some other settings, added whole new sections. And each time I made a change, I kind of said, “Okay, I’ll figure out later how my photos fit into this new site.” I kept pushing it off.
I needed the “words” to work before I figured out how the photo and videos would work. Or if they even would.
That’s no small issue. While there are huge advantages to having everything in one site, my site has grown. It’s quite large. It has a LOT of moving parts. And the more I push in certain areas, the more I expand my content, the less functional it seemed having everything together.
A few weeks ago, I had to bite the bullet and decide. Was it going back to having two (or more) sites for PolyWogg content, maybe one for my HR guide, a separate one for regular blogging, a separate one perhaps for photos? In the end, I reframed the question. What were the REAL obstacles in having it all on one site?
The final analysis brought me to two pain points:
Navigation
Branding
Content management wasn’t the real issue. It was that I have a lot of content that I want to group together but branding it doesn’t really work with my standard “PolyWogg” headers. And navigation amongst the sub areas is too hard when you only have one pull-down menu for that category with a lot of sub-sub-sub-menus.
Again, as with all things in web developer, there was another option besides a separate site. I could, in theory, have separate headers for my different content as well as separate menus. There would be one master menu for the site, but once you got into more granular areas, you would move to a wholly different menu too.
Except I had tried this on multiple occasions, my theme is SUPPOSED to be able to do this, and I’ve never been able to get it to work. I’ve tried other plugins, nada. But this is what I WANTED. Maybe I could bang my head for a few weeks and see if I could cobble together a solution.
Okay, step 1, reach out for theme support. See if they had suggestions as to which other plugins would work well with the theme to do exactly what I wanted. Or tell me how to make it work with the theme. I’ve had some luck with them in the last year tweaking my theme, so I was willing to give it a go. I posted my question, aaaaand I crashed their site.
I’m not kidding. I literally crashed their support site. They fixed it and went, “Huh, what happened?”. I told them I had been posting a question, they double-checked the log, and sure enough, it was my account that killed it. My account is somehow corrupted (they don’t know how or why), and my posting killed them. They’ve tweaked it so that I can’t do it again, but my acct is still messed up somehow. I can use it, but well, I get some weird screens that others don’t get. No worries, I’ll survive.
Except in the meantime I figured I would see how far I could get on my own down this rabbit hole. I went to my theme. I enabled the features. I went to my test page, switched the header to the proper one, no change. Yep, I remember that outcome. Went back to another sub-page, made some more tweaks, misread an option, set it, reset, now NO header. WTF? Oh. Oops, misread it. Okay, reset that option, found two others that seemed to make sense that I haven’t noticed before, might be new, retested. And my header changed. All of my branding changed for that one sub-page with 2 minutes worth of work. Holy smokes.
Okay, don’t get cocky, I thought. I went to the menu area. I tried to create a new menu, copying over my old one. Told it not to put it ANYWHERE, just a dummy menu. Went in and deleted some stuff just so I could see that it was different. Went to the page that had the new header and told it to show the new menu on that page, not the regular menu. Reloaded. BAM! All of my navigation was changed for that one sub-page with 2 minutes more work.
OMG.
I did it. Exactly the way it is supposed to work, and I’m 95% sure, exactly the way I had tried it on previous occasions. But I don’t care now. It works.
Which meant I could keep my single site. Which means no separate setup. Nothing to stop me from using my existing site. All I had to do was decide on a consistent format to my layout and design for that sub-area that wouldn’t bite me in the butt later.
Because I’m not talking some small site. The average site in WordPress for people using other galleries is maybe 1000 photos. Sites that run full WooCommerce and sell products frequently don’t have more than 1000 items in their site. For me? We average 2000 photos a year, of which I post about a third. We broke a thousand mid-way into year 2, I’m over 2000 by the end of year 4, and I haven’t even included all the photos from our wedding events that year. Including the honeymoon section which is huge.
Long term, I’m estimating somewhere around 20K photos just to get caught up to now, although that may top 25K. I know professional photographers who don’t have that many. AND I haven’t even got to what I want to do for astrophotography images.
Rebooting the gallery
Since I had already uploaded the photos for 2005-2008, I didn’t have to do much to “fix” those galleries. I renamed a bunch, I changed the look and feel from an old template being deprecated to a new one, tweaked some inconsistencies here and there, and added a new video section that works really well, so I’m generally “good to go”.
Previously when I played with the first 4 years worth of gallery, I had to spend a lot of time getting them up and running. Maybe one gallery a day. I just did 32 galleries in about four days, one year per day, generally about 2 hours work while I was editing other things.
I’ve even managed to get past my previous point of progress (2005-2008), completed all the old galleries for the wedding, and I’m finally back into the truly “new” ones for being part of WordPress. I had reached 2011 at one point with SmugMug, I think, but I’m pretty happy with my early rebuild. I have a full workflow figured out, complete with Mylio as my software, and it is giving me the confidence that I have finally “turned the corner” on my go to solution. Four years down, thirteen to go, albeit the next thirteen won’t be anywhere near as fast. And alas, 2009-2011 is redoing old work. At least I’m doing it properly now.
Just don’t ask me about astrophotography yet. I don’t know HOW I’m going to organize that stuff.
Overall though, apparently the “seventh time was the charm”…I have won a decisive battle, but the war rages on.
I mentioned in my posts about my goals for 2021 that I want to do a fair amount of work on my photo gallery (Setting goals for 2021 – Part 5: Computers, Website, Blogging, Writing, Media and Photos). But in order to do that, one of the things that is a challenge is ensuring that each gallery (say, a given month of a year) is set up consistently each and every time. That’s not a “small” step of consistency, but one that starts from the very beginning. So let’s talk about the different stages of the workflow.
Stage 1. Taking the photos
My photo gallery is made up of photos that come to me from multiple sources, and I need to be able to tell them apart for the purposes of managing. I can manipulate the filenames after the fact, but it would be easier (as my friend Matt suggested) if the filenames themselves were a bit more process-friendly. So let’s look at what those FNs would look like:
Andrea’s iPhone –> IMGxxxx – YYYYMMDD – AH
Paul’s iPhone –> IMGxxxx – YYYYMMDD – PS
Jacob’s iPhone –> IMGxxxx – YYYYMMDD – JH
DSLR –> IMGxxxx – YYYYMMDD – DSLR
Point and Shoot –> IMGxxxx – YYYYMMDD – SC (* for small camera)
Screen grab –> IMGxxxx – YYYYMMDD – SCR
Tablet –> IMGxxxx – YYYYMMDD – TAB
Now, some of those can be assigned at time of capture, while others will have to be assigned at time of file transfer. Depends on the original tool. So the workflow bifurcates after this step, but the step here is the same:
Take the picture
Stage 2. Managing the photos – Pre-processing and pre-sorting
Managing the photos is initially quite easy. I have to copy them from the device to the same folder on my PC, and I do it in four stages depending on the imaging source (copy to the hard drive, copy to the sorting folder, simple sort, and convert videos).
For Andrea’s iPhone and the Point and Shoot camera, the photos get transferred to her desktop and then copied onto a USB thumb drive.
For Jacob’s iPhone, the photos get transferred to his laptop and then copied onto a USB thumb drive.
For my iPhone, the DSLR, and the tablet, all the files are synched onto my desktop into a Synching folder.
Any files that are not already renamed properly can be renamed at this stage.
For the final step, all eight original sources are then moved into a sorting folder where I sort them by month and day or event into relevant folders. Since lots of shots are grouped together, I create a file folder structure such as:
YYYY / MM-Month (general folder as the upper level folder)
YYYY / MM-Month / DD (Event title) like “14 Kayaking at MEP’s” or “29 Birthday party”
YYYY / MM-Month / Extras as a folder for ones not being used in the gallery (often mistakes or just not meant for public, like a photo taken in a store of something I want to buy)
YYYY / MMb – Special – Special event for the month (#1) like a party, trip, etc. where a large number of photos warrant their own gallery
YYYY / MMb – Special – Special event for the month (#2) like a party, trip, etc. where a large number of photos warrant their own gallery
YYYY / MMz – Blog posts for photos that I’m going to eventually upload to the website to include in various blog posts but are generally not interesting enough on their own to include in “general photos”.
For the first level of sorting, I move everything into the relevant folders. It may mean, for example, that I have a folder for a big trip that has photos from multiple sources in it, and in fact, I usually do have at least 2 sources for various daily events.
For the second level of sorting, I convert all MOV format videos into MP4 format (suitable for the web) and move all old videos into the EXTRAs sub-folder.
Copy from the device to the hard drive
Copy from hard drive (potentially via USB flash drive) to the sorting folder and rename any if necessary.
Do a simple sort by event and dates
Convert videos to MP4
Stage 3. Managing the photos – Advanced sort in Mylio
I use Mylio as my image manager, and I do four steps in Mylio.
First and foremost, I import all the images from the sorting folder including the directories I created. When they arrive in Mylio, they are in a sorting folder too. I basically go through and move the quality images I want to use for each event into the MONTHLY folder (such as 2021 / 01 January). Extras that I’m not using, such as the secondary or tertiary photos of a group of ducks, I move into the EXTRAS folder. I also do a quality sort on the videos, special events, and folders of photos I intend to use for my blog posts.
In some cases, I may decide to edit a video or photo to make it suitable for sharing, in which case I make a copy and edit the copy rather than the original. I usually do this in another program beyond Mylio and then reimport the edited version.
Once I have a set of photos and videos for a given month (for instance, 2021 / 01 January), I run facial recognition on the “good” photos (there are too many photos to worry about doing the Extras too) and I let the computer do most of the work to group them and guesstimate who is in the photo. After the first few hundred of a given year are posted, the rest of the guesses are usually pretty accurate on the first attempt. I then add metadata to the files. This includes a name and description for the photo (identical as it is used differently in WordPress), something short, and some keyword tagging that includes year, city, event, and any people in the photo who were tagged in the Key Words. Finally, I save all the metadata to the image file.
Finally, now that the photos are all sorted, named and tagged, I move it from a sorting folder into my full folder structure for the Panda Photo Gallery in Mylio which generally has the structure of FAMILY / YEAR / MONTH.
Import images into Mylio sorting area
Do advanced quality and photo selection, filing the rest in EXTRAS.
(Optional) Edit any photos or videos that require tweaking
Facial recognition
Add metadata (name, description, tags/keywords)
Save metadata to image file
Move from sorting folder into a full folder structure
Stage 4. Uploading to WordPress
Up until this point, most of the file management stuff is just simply a good process / workflow for keeping my photos organized and filtered for quality. Now I look at the parts of getting it on the website.
Initially, I create a page to hold the gallery (while this could be done later, it saves a step in the gallery creation process) and assign it a name such as 2021/01 January, and insert two default items — a blank photo gallery and a blank video gallery. I save the page, but leave it in edit mode.
Then, working by folder (such as 2021 / 01 January), I create a new GALLERY called 2021-01 January in WordPress. This opens an upload area, and using Mylio as my initial interface, I upload all of the good PHOTOS for that month/event. Since I already populated the metadata fields, the upload puts everything into the WordPress fields for me. This completes the step of uploading all the PHOTOs. I can then edit the GALLERY description to describe the various events in the monthly folder (copying the description for later use, like a descriptive table of contents), add the link to the page created above (the step that I saved allows me to do this now), and add the GALLERY to the appropriate ALBUM (such as the year, 2021).
Then it is time to upload Videos, if any. These have to be done in the Media Library, and using Media Library Assistant, I save them to a separate sub-folder usually called YYYY-MM. Now that all the files are uploaded, it’s time to go back and edit the page I created earlier.
Each Gallery page has six components to edit:
The Page name, if it needs to be tweaked from the standard page name (usually YYYY-MM for months but could be YYYY-MM Special – Trip to Mexico);
A manual breadcrumb that I’ve created to allow the viewer to go one level higher easily;
A description of the gallery (same as what was already entered in the Gallery Page, just pasted here);
The blank gallery block to choose which gallery I want to show, and to change the order of photos if needed;
The names / description of any videos that need to be linked; and,
Linking to the videos themselves.
Finally, everything is saved and the page is previewed to make sure everything works, and the page is published. The link is then shared to FB along with the gallery description.
Create a page and edit the page name, add a blank photo gallery block and a blank video gallery block, and save in draft mode;
Create the new gallery, upload all the photos, edit the gallery for gallery description (and copy for later) and link to the page created above;
Add the GALLERY to the right ALBUM;
Upload videos and sort into sub-folder;
Go back and edit the page for page name, manual breadcrumb, paste the description, choose a gallery in the blank gallery block, edit the names / descriptions of the videos, and link to the videos themselves;
Save and preview/test, then publish;
Share link with FB.
Stage 5. Backups and further usage
Mylio automatically does a backup of all photos to a secondary location, and long-term, I want that to upload to the cloud too. Later, I do a separate backup of all my files to off-site storage.
At the end of the year, I also take all the “GOOD” photos and put them on a USB thumb drive for Andrea to weed and use to make a Photobook. Once she’s done, I save the final photos back to another folder labelled PHOTOBOOK. If there are any really good ones for the year, we also use them in Calendars, New Year’s letters, metal prints, and an e-frame.
Backup to secondary location/vault;
Backup with all files to offsite location;
Create a small subset each year for Andrea to use for photobooks, calendars, New Year’s letter, metal prints, and e-frame;
Copy subset back to a folder called PHOTOBOOKS.
And then, finally, I’m done. Whew. So let’s look at that workflow all together so I don’t miss anything each time. I’m also going to copy it into a PowerPoint print-out so I don’t lose it. Nineteen steps that I have to do consistently every time or something gets messed up.
Take the picture
Copy from the device to the hard drive
Copy from hard drive (potentially via USB flash drive) to the sorting folder and rename any if necessary.
Do a simple sort by event and dates
Convert videos to MP4
Import images into Mylio sorting area
Do advanced quality and photo selection, filing the rest in EXTRAS.
(Optional) Edit any photos or videos that require tweaking
Facial recognition
Add metadata (name, description, tags/keywords)
Save metadata to image file
Move from sorting folder into a full folder structure
Create a page and edit the page name, add a blank photo gallery block and a blank video gallery block, and save in draft mode;
Create the new gallery, upload all the photos, edit the gallery for gallery description (and copy for later) and link to the page created above;
Add the GALLERY to the right ALBUM;
Upload videos and sort into sub-folder;
Go back and edit the page for page name, manual breadcrumb, paste the description, choose a gallery in the blank gallery block, edit the names / descriptions of the videos, and link to the videos themselves;
Save and preview/test, then publish;
Share link with FB.
Backup to secondary location/vault;
Backup with all files to offsite location;
Create a small subset each year for Andrea to use for photobooks, calendars, New Year’s letter, metal prints, and e-frame;
