The reorg project has been in full swing for some time, and part of the challenge is the mish-mash of items that are stored in multiple places:
Tools like small hammers or screwdrivers in the garage, in the first-floor alcove near the front door, in the basement where I have my desks now, and in the office upstairs;
Books in the bedroom, office, basement, Jacob’s bedroom, and even the guest room;
My clothes in the bedroom, closet, guest room, and front hall closet;
Housewares / camping items in the laundry room storage, main basement, alcove, garage, and basement;
Astronomy stuff in the alcove, basement and garage;
Sports stuff in the basement and garage;
Power tools mainly in the garage, but a couple of smaller items in various parts of the house;
Movies in the bedroom, basement, laundry room storage, and first floor;
Electrical stuff in the main basement, laundry room storage, and garage;
Video game stuff in the laundry room storage, main basement, and first floor;
Photo books in the office and bedroom;
Games in the office, bedroom, family room, living room, main basement, garage and laundry room storage; and, of course,
Papers in almost every place.
In most cases, that mish-mash made some sense on a case-by-case basis. But since I want to do some purging, it’s kind of hard to purge a collection of screwdrivers, for example, if they’re in three different locations. I pretty much have to put everything close to its final resting place before I can do a proper organized purge.
I have frequently called it a domino problem. For example, organizing some of the housewares and camping stuff is a bit painful when it is in at least two different locations, and moving it from one to the other requires me to move a bunch of other stuff out of the way to make room, which in turn means making room somewhere else for THAT set of items. Dominoes.
But paralysis by domino is a poor excuse / rationale for not being organized. And quite frankly, I don’t have the luxury of some of it anymore, not if I’m going to get my office working and organized properly.
I’ve already made a GIANT dent in the office and basement, Andrea has done a lot of stuff in the family room and toys from the basement to the office storage area, and we’re pretty close to being able to say things are all grouped somewhere. Like, “Okay, this is ALL the audio-visual stuff in the house all in one place, what am I keeping and what am I purging?”. It’s still not going to be easy on some of it to decide to get rid as much as I hope to purge, but regardless, if I’m going to finish by the end of September, there are two giant areas left untouched. I need to dig into them just to figure out what’s there even, beyond the general idea I have already. I still have the first-floor alcove and the garage.
Today, Andrea and I tackled the garage. Most of it was going through about 25 small shoebox-sized plastic tubs that had everything from screwdrivers to drillbits, from Allen keys to tape, and from screws to brackets for shelving. Some of it went GREAT. It was easy to see where things went, easy to group them, easy to dispose of some stuff.
Other areas were not so great. Like, for instance, a bunch of things I have for curtain rods. Which I would LOVE to get rid off, but Andrea and I have at least four or five places in the house where we want to change the existing window coverings. So it seems premature to purge curtain rod holders before we know what we’re doing for those locations.
I also haven’t quite figured out what I’m doing with multiple sets of sockets and wrenches that are all jumbled together. Most are labelled well; some are not labelled at all or worn away. I’m tempted to keep the newest and/or best quality set, and ditch the rest. I have this vision of two small tool bags, one in the garage and one in the house, both relatively identical with a good set of each size of screwdriver, for instance.
But as I said, I need all the sets together in one place to then start dividing them up properly. Today was phase 1 of the garage, and we went through two large shelves of small items. It took about two hours and wiped me out. While it didn’t empty large areas of the garage, it was the major parts that take time since we had to open each little box, sort through it, decide what to keep or purge, and even for the purging, seeing if it was something that Andrea could give away on the local “free” Facebook recycling groups or if it was just garbage. Or even if it was garbage, was it e-waste or chemical or recyclable or just plain garbage? A thousand little decisions and it’s exhausting. The electrical work is going to be worse as most of that will be just me going through it, today I had Andrea helping me decide.
The funny thing is it likely looks worse after we were done with lots of piles of things all over the garage on tables here and there.
Phase 2, which I’ll do sometime in the next two weeks, is also going to be a bit brutal. A lot of “big” things to decide if we’re keeping or not. Tarps that we found useful for x or y purpose, but do we need all of them? Some toys of Jacob’s in the garage … do we keep toys for going to the beach to build sandcastles? Which we haven’t used in 5 years probably? Are we keeping all the balls we have looking forward to maybe having a pool next year to play with them in? Or do we purge them now?
And a giant set of questions for me around what I’m doing with my astro gear. I have an option to build a small enclosed parking area in the garage to put a rolling wagon in that would let me haul all my gear to the backyard relatively easily. But if I take that option, I would keep some nails and screws, plus a bunch of wood, AND I’d have to get rid of the workbench. Or I can try to fit it in where there’s a wardrobe now, maybe even using some of the wood from the wardrobe to build the cover. Or do I scrap the covered idea, and then I can get rid of the fasteners AND the extra wood that is there. I am definitely not going to be building any more shelves, and so I have a bunch of shelving that can be purged. Unless, as I said, I build that astro box. Sigh.
25 days left in September, so it will be crunch time.
And these are not the only things left to do, just some big areas on the list that have to get going so I don’t end up having to repeat purging steps.
Today I choose to tackle the first phase of the garage work, with two other phases to go. We (Andrea and I) made good progress, but there’s still a lot to do.
I have a lot of digital music on my computer, and a subscription to Apple Music. So just about anything I want or need is available digitally. So we’re purging our CD collection. That’s a separate issue altogether, and while I’m happy to donate them wherever will take them (unfortunately the library is saying no to everything at the moment), it also gives me a small nudge to organize my digital collection.
Putting the various pieces together
Based on the various reading that I have done, having a well-organized and functional music collection involves five main pieces:
Storage
Backup option(s)
Tool(s) to manage multiple formats
Organization of the files
Playback and sharing options
Storage:As I mentioned, my storage is almost all digital at this point. We’ve kept a few CDs that we are loathe to part with, but I suspect that is more a transitional collection. Eventually, we’ll dump them too, simply as we’ll find it easier to playback the same music on devices that don’t require CDs.
When I recently upgraded my computer, I put in an extra-large hard drive, and some of that was to hold my music collection. I wouldn’t say my collection is enormous or anything, about 150GB in total. Some people literally have terabytes of music, with some files dating back to the *cough* Napster days.
I don’t judge, whatever floats your boat, man. 🙂 A former boss of mine was into classical music, a hard-core audiophile, and he spent $8K at one point on upgrades to his house to improve the quality of the playback experience in every room. $8K and that was JUST FOR THE WIRING.
Everything is stored on my harddrive, easily accessible.
Backup options:Most articles talk about having a backup option, singular, and that just seems ridiculous to me. You have an enormous collection of music, probably hundreds of hours just putting it together, and you’re relying on a single backup solution?
My backup solution starts with the word Apple. In my defence against the black arts that Apple performs on hard drives, I do NOT let Apple manage my main music collection. It thinks it does, sure, I let it have the My Music folder as it’s primary work area by default. But I have a totally separate folder called MUSIC MASTERS that has all my original files in it. If I want something added to Apple Music / iTunes, I copy it over. Apple has spent a lot of time in the last few years to stop its software from overwriting people’s original files, but it is not foolproof. While I don’t have million-dollar recordings or irreplaceable versions of anything, I am not letting Apple touch my originals. Ever.
So my first backup is actually Apple while my ORIGINALS are stored a full folder away. Then I copy the whole double set of files to my regular external hard drive regularly, and then all of my drives to external storage. While I am more worried about my photos than my music, it all gets dragged along to the big backup in the sky (although not literally the cloud, I’m mixing metaphors here, but I will have a cloud option done by the end of September too).
Tools to manage multiple formats: I looked at a lot of different music players over the years, both in terms of software on my machine as well as physical tools. Sony had some music management software that went hand-in-hand with their MD players and walkmans (called Sonic Stage/Sound and NetMD). RealPlayer was in there for awhile, as was WinAMP way back in my DOS and early Windows days. OpenMG Jukebox. A player for my Coby MP3 player. A few options, to say the least, and that doesn’t include the 1000s available for download.
But, over time, I keep coming back to Media Monkey. It isn’t the slickest of interfaces, often feeling more DOS-like than full 21st century GUI. But it handles all the file formats I use (more on that in a minute), handles playback fine, and other than a slow opening where it re-reads all the files in the sub-folder structure, I trust it well enough to let it see my MUSIC MASTERS folder. For one simple reason.
It is more of a file manager with extensive music tools than an extensive music tool with basic file management. For example, over on the left side of my screen, I have a regular file tree with all the folders shown. I can browse them like I was using File Explorer or Xplorer2 (not actually, but similar). It gives me better content viewing of the files once I get to the file structure part, and it’s not perfect, but it’s the best file-based interface I have seen. Unlike Apple Music which has the slickest interface for browsing, but almost hides the file structure behind its menus.
But there is an enormous rabbit hole that I’m almost sidestepping here — if you have a tool that will manage multiple formats, you first have to recognize that there ARE multiple formats and understand to some extent the pros and cons of each format.
I don’t pretend to do that. I get that there are huge communities out there that will debate true lossless formats, would never accept anything less than AAC or FLAC or AFLAC (wait, I think that’s the insurance company, scratch that one). But unlike my former boss, I really can’t tell the difference acoustically between an MP3 recorded at 320 bps or merely 192 bps, let alone the lossless levels of other high-end formats. I’m fine to have some in that format, but if it’s in mere 192 or even 128, I’ll take it.
So most of my music is in some form of MP3 format. Regardless, Media Monkey can handle just about anything I throw its way. It merely needs the input. And if I find something it doesn’t handle by default? I have lots of file converters that will pre-process it for me.
Playback and sharing options: I skipped over the organization heading as that is a separate rabbit hole all on its own. For playback and sharing, though, I guess I should distinguish between several layers.
The first layer is simple playback on my computer. Media Monkey can handle that, as can Apple Music or a host of other music players. Nothing complicated there.
The second layer is playback around the house, and I confess there I’m not well-equipped. I have some ideas, some devices, but I generally rely on internet playback rather than casting about the house or direct access to my computer. I have some tools to do basic upgrades in the coming month, and it will likely be sufficient. Not by an audiophile’s standards, but for my basic needs.
The third layer is playback while mobile, and that’s where things get interesting. I have Apple Music, with a full family subscription, and since all three of us have iPhones, it’s a good investment. Eventually, ALL of the music will be available on my Apple account and shared with all 3 of our accounts, but after I tried letting Apple upload everything willy-nilly, I realized what a crapfest that would be. So I deleted it all and started over. I may have to do the same with Google Music, Amazon Prime Music, and YouTube Music. Why do I have those other services running? Because I just use their free option, and sometimes things I want are not on Apple Music. So why not? All it costs me is a bit of time to share them and upload them.
My real playback challenge though is simply 20-30 feet from my main computer. I like to listen to music while I work out, preferably LOUD and distracting to put me into a zone, and I will soon have three mini-areas set up in my basement for exercise. Well, soon being in the next month, if all the phases hold. But what do I do if want to put on headphones and just dance around? I know, I know, you didn’t need that image.
I have two basic options — corded or wireless. But my wireless runs off bluetooth, and outside of my phone setup, none of the options downstairs, not my computer, not my stereo, nada has built in bluetooth connectivity. I have a tool that should help with it, and I have no idea where I put it. Alternatively, I can run something from my main stereo with headphones using a big ass jack, kicking it old school, but my stereo isn’t really set up for digital input either. Overall though, wireless would be better, I just need to set it up properly. I have good headphones I can use for the work out or going truly mobile, but if I just want to lay on the couch and veg with music on, I’d like something a bit more encompassing than wireless buds. But I’m not paying for the high-end stuff, my ears can’t tell the difference anyway. It’s just that I can’t be blasting tunes when the rest of the family is home. The floorboards are just too thin and not much of a sound barrier.
Sharing is more complicated. Generally, it is about sharing with Andrea and Jacob, and that is about sharing the digital files themselves. That will be solved through Apple Music, so not a giant challenge. But I would also like to be able to share the playlists and some music reviews on my blog, right here. I ran a test with one I did earlier this weekend for an old album, and for some reason, the playlist will not share properly. It shows 8 of the songs, but not all 10. I assume it’s the difference between sharing 8 songs that are freely available in subscription mode and 2 that are separate purchases or something, but I’d be fine if it JUST shared a preview of the songs. I’m not trying to share the actual music, just the list with some links for people.
Which takes me to those other streaming services. If in the end, it is easier to share a playlist from there or share playlists across platforms through Media Monkey, I’ll do that. Because I have a HUGE project that I started 4 years ago, and it will take me a long time to do. But it requires me to have the ability to share playlists, unless I want to do a LOT more work manually. Pass. So I still need to figure out a reliable workflow for sharing the lists, not the actual files themselves.
The rabbit hole of genres
Genres doesn’t feel like it should be a rabbit hole. The archivist in me, the analyst in me, the music lover even, all scream that order should be rather simple. And if it is something like “Christmas music”, that seems like a no-brainer. True, I might argue that “Grandma got run over by a raindeer” isn’t exactly the same genre as Bing Crosby singing “White Christmas”, but so what? It’s all primarily Christmas music. Or even “holiday music” if you want to be a little more inclusive and throw in the dreidel song or something.
But even in my old setup, the genres were a problem. Sure, I could have a simple folder for Classical. And a category for Movie Soundtracks. But what if a soundtrack used mostly classical music? Well, if it’s a soundtrack, that would be clearly with soundtracks to me. If the “nature” is classical and classical only, it would go there. But the main impetus for having a group of music around a movie is the soundtrack itself, tied to the movie, not a composer, so seems simple.
Then I come to someone like Alannah Myles. She’s not clearly “rock” although some might think so. Including herself at times, apparently. Clearly “pop”. Except Black Velvet is not really “pop” per se. Certainly not uptempo normal top 40 pop. Some of her other albums even go pretty close to country. Hmm.
Well, that’s an aberration, right? So what about someone like Shania Twain. She was clearly country until she went to pop country and a bunch of songs crossed over to simple pop. On the same album that had clear country ballad tunes. Sigh.
Okay, let’s start with some old 60s rock. Although a lot of 60s rock was really 60s pop, like the Beatles. But I certainly wouldn’t put the Beatles in the same genre as AC/DC or even Bob Seger. It’s all spectrum stuff, some argue. Huge swaths that are rock and roll in all its forms, and sub genres for everything else!
Oh, dear lord.
I shouldn’t despair, some of the basics are fine for me:
Rock
Pop
R&B
Soul
Reggae
Blues
Jazz
Country
Folk
Vocals
tbc
tbc
tbc
tbc
Classical
Soundtracks
Musicals
Comedy
Christmas
Kids
Radio shows
My collection falls heavily in the first two categories, rock or pop, from the first column. I’m not sure I’m sophisticated enough to separate out everything from R&B, Soul, Blues, Jazz and even a few types of Reggae, they blend together at times for me. Or if it even matters. I struggle with some of column 2 that a few artists are sure not pop or rock, but aren’t really folk or country, and yet they do have very strong vocal components. Column 3 goes back to normal, with some very obvious categories that seem fine.
So I reached out online to see if any of my friends were closet anal-retentives when it comes to musical genres and filing, and they fell into three giant camps.
Camp A is the “I’m not organized” category. Pretty common, lots of people have their collections stored all over the place.
Camp B is great if you want to go into a sub-genre world. One friend noted that she has sub-categories for “Surf music, British Invasion, Psychedelic rock, Glam rock, Classic Rock (70s), Hard Rock, Soft Rock, Punk Rock, New Wave, Grunge, Britpop, Indie rock” and anything after that is too narrow to need a niche. I’m not even sure I could name a band in each of those categories although I like some of it. I certainly have surf music, British invasion, soft rock. Classic would be hard to nail down evenly. Not so much the rest.
Another friend noted his genres go in a different direction. He separates his physical collection by geography, (Canada, USA, Europe/World), old school, soundtracks, comedy and compilations. It’s interesting, I have some overlap, but geography doesn’t excite me. I doubt there are many times I would think, “Hey, let’s listen to some American music” as something that would help me find it.
And that type of search is where Camp C comes in. For them, it is more about mood. So, for example, a up-tempo collection for partying or dancing, or a relaxing mellow collection for Sunday afternoon. It is a huge camp out there that does that, and I see the attraction. But to me, that is what playlists are for, not how you save your music? Dunno. Might be the archivist in me.
And therein lies the rub. Some of what I’m dealing with is the mental side of “what’s the most logical way to organize it”. First and foremost is the band, then an album, then the songs. It seems natural to me. Some might do years, but I feel the album is the proxy for year. And if I just want to find a random song, I can always search. But for backups, organizing, almost nothing seems more fundamental than the band itself.
Except, perhaps, genre. Does it make sense to have an alphabetical list of every band in a general folder or is it like my physical collections where I put all my country music together, all my pop, all my older-style rock. Which was fine when it was a handful of CDs. Now that I have a much larger digital collection, and things are more likely alphabetical, does it make sense to put the Beatles next to AC/DC? Probably not.
Sigh.
Is there an off-the-shelf solution?
In an ideal world, I wouldn’t have to think about this. Someone else would have already solved all of this already, someone IN THE BUSINESS, who knows the difference between small shades of nuance in rock genres for example. A professional. Or a business.
Like Apple perhaps. Yes, they have default categories:
Alternative
Blues / R&B
Books & Spoken
Children’s
Christmas
Classical
Classical Crossover
Comedy
Country
Dance
Easy Listening
Electronic
Folk
Hip Hop / Rap
Holiday
House
Industrial
Jazz
Karaoke
Metal
Musical
New Age
Original Score
Pop
R&B / Soul
Religious
Rock
Singer / Songwriter
Soundtrack
Techno
Trance
Unclassifiable
World
Thirty-three default categories to arrange everything. Sounds great, right? Except it does nothing for me that I didn’t already have. It puts Blues and R&B together and then puts R&B and Soul together. Same problem I already had. It separates folk and country, sure, but doesn’t solve my “vocals” problem. Classical crossover? What the HELL is that? Popular classical, like Rachmaninov, or the classical songs that show up in movies? Or Beethoven’s Fifth, disco style?
Easy listening I guess is meant to be the non-rock, non-pop, non-folk “soft stuff”? Maybe I put the vocals in there. Or maybe it is just soft rock. I will probably never have anything in the Alternative, Books, Classical Crossover, Dance, Electronic, Holiday, House, Industrial, Karaoke, New Age, Original Score, Religious, Singer / Songwriter, Techno, or Trance. Taking me down to 18 possibles, although Easy Listening, Hip Hop / Rap, Metal, and World likely aren’t topping my list either. Say, maybe 14 categories. Is that better than just 6? Or my original 17?
It’s a start, at least. I’ll have to check out the other online streaming categories, but even they have started to go by mood in a lot of places.
As part of my #50by50 posts, I repatriated all my videos and pictures from SmugMug, threw them into Piwigo, and (mostly) completed a good layout and design for my online photo gallery. I had tried integrating directly into WordPress, but the biggest and best (relatively speaking) gallery called Next Gen Gallery just didn’t play well with some of my other plugins, and I couldn’t get it to work right. I tried various other WP tools, but nothing was jiving for me. Piwigo worked, I found some themes I liked, I tweaked some stuff, called it a day. Then proceeded to put a LOT of time and effort into uploading 12755 photos and videos of various types and sizes.
I made it as good as I could, but it was far from “perfect”, if there is any such thing. For example, Piwigo likes to play with different size images. So it would take the original ~13K photos and make a thumbnail for each one. Plus a medium size. And a large size. Which means ~13K photos suddenly becomes ~52K files on the site. Plus the Piwigo install itself…plugins, core files, themes, etc. Call it another 3K in admin files, and I’m at 55K for the number of files. Which isn’t a problem on the one hand — my account comes with unlimited storage space. Great! Except there’s a small caveat to that unlimited storage space. It only allows 200,000 nodes which are basically file markers. 200K nodes = 200K files. I’m only at 55K, but the wrinkle?
That’s just the gallery. I also have AstroPontiac, ManagementConsultingServices, and oh, yeah, all of POLYWOGG.CA i.e. this site within the 200K too, with separate full installs of WordPress three times (that’s another story, but still). Which at one point put me close to 150K nodes. As I continue to add and upload stuff, that “margin” starts to shrink. Not a problem “yet”, but I’m looking at expanding my online presence soon, and Piwigo is taking up a lot of nodes.
Enter a new wrinkle — or two!
My hosting provider recently migrated a whole bunch of accounts to new, larger, faster servers, and my account went with it. But after it was done, for some reason, part of my WordPress install and part of my Gallery were no longer working. This is not an uncommon problem, actually. One of the downsides of running multiple installs on my server is that a couple of key files, mostly related to security, all reside in the same directory. So three copies of WordPress and one copy of Gallery all want to play in that same directory, and they don’t all know how to play nicely. When something changes for one, it can — and does — present challenges for the other installations. The three WP sites got along fine. But my Piwigo gallery wasn’t liking the new server setup.
My hoster fixed it, great. Then it broke again when something changed. So they fixed it again, great. Then it broke again. So they fixed it a third time, and it was still broke. A fourth time, still broke. A fifth time, fixed and stayed fixed. But it required a couple of tweaks that are not optimal for site operations. Not mission-critical problems, but a small design challenge, and likely to cause me problems down the road with other plugins and operations.
So, I reached back into my blog, pulled up my musings from earlier about different plugins to replace Piwigo with the idea of trying to fully integrate into my blog, and of course came across Next Gen Gallery again. Over 900,000 sites use the plugin for galleries. And yet again, I thought, “Why won’t it work with mine?”. So I gave it another go, expecting it to fail but thinking maybe this time I could devote some time and figure out what the conflict was and fix it.
I installed NGG, activated it, tried a test gallery, worked perfectly. Wait…what?
Setting up NGG for my gallery
Yep, it works now. I think mostly because I’ve switched security plugins and now it likes my configuration. Or at least doesn’t hate it. Well, that changes things. I started playing with it, a few limitations that I can live with, and I decided to go for broke and buy the pro version. Also works perfectly. Relatively anyway.
Sure, I have to tweak it for setup to match my themes and blog, as I would with any plugin. There are a bunch of gallery themes, none that work well enough to replace my overall theme, so I can ditch those. There are also layout templates, some basic, some pro, and lots of tweaks that each one can do. In the end, I really like a first page which shows thumbnail images. My favorite is called the Pro Thumbnail Grid, lets me put a legacy caption below each photo, space them out more or less grid style, and also make them fully responsive (i.e. on small screens, you get 2 images across; on my wide-screen, I get 4; on mobile, just 1). I can set a default for most of the settings, change the colours to match my blog’s theme a bit more, etc.
And then choose from a handful of different lightbox settings (i.e. the way it looks when you click on a thumbnail and it opens the pic into a full image, complete with caption, social media sharing icons, and a place to comment on the picture if you want). I had to do styling tweaks on both to get the result to look the way I wanted it to look, but one of the benefits of having the pro version is that it comes with support. So I asked questions of the developers and they told me how to style some of my unique tweaks. Which then led me to figure out some of the tweaking on my own, a sense of accomplishment that pushes my ego button pretty hard. I was pretty self-satisfied with my initial progress, particularly as it has me doing some CSS style sheet tweaks that I’ve never really done before at this level. 🙂
There are still some formatting bugs to work out such as some styling of breadcrumbs on an internally-generated virtual page. I also found a great alternative layout to use for my astronomy photos. It includes EXIF data (camera setting info), which is helpful to see with each picture. I haven’t fully styled that page, but should be only minor tweaks once I get to my astro photos.
But wait, there’s more
One of the ongoing challenges I have always had with my images is that a lot of the data is manually entered with the pics online. So all of my so-called meta data for captions, folder names, descriptions? They exist only in cyberspace in the database of the apps I’m using; the pics themselves do not include those descriptions. Which means when they went from desktop to SmugMug, they all had to be re-coded manually. When the photos went from Smugmug to Piwigo, a small percentage of the data went with them, but most had to be manually re-entered. Now that I’m going from Piwigo to WordPress, the spectre of potential recoding rears its ugly head yet again.
But as I went through photo editors last year including looking at photo management options, I tripped over a program called Mylio. It is not the best editor by far, but it has an advantage over others. It allows you to directly edit metadata, embed it in the photo so it never has to be updated again, and when uploaded to NextGen Gallery? It can read the info and display it. Including not only captions to go with each photo but any extra “tags” I put on the photos. Sure, there are other programs that do that, but can they do it in an easy to edit “group photo” page? And more importantly, not for the blog, but for self organization, can the others do decent facial recognition? No, not very well.
Yet Mylio was one I tried before, and at the time, I set it aside for later when I plan to process some photos from my mom. But if I’m going to the trouble of fixing all the metadata — and doing it right so I never have to do it again! — then I might as well have the biggest tagging aid working properly too for my own photos. Booyah!
But wait, there’s less
While having Next Gen Gallery working and using Mylio to organize the photos before uploading are great, there’s always a catch, right? Of course there is. NGG doesn’t manage videos.
Crickets. Chirping.
So? So, I have a fair number of videos of Jacob, for instance, embedded in my current gallery. Which works REALLY well, and I like it. Alas, NGG won’t handle video. And 18 months ago when I ran through a whole whack of gallery options, if it didn’t have an option for video, I killed it right away. 18 months later, I’m not as fussed about that. I can find work arounds, as long as I have a really decent photo gallery working that is fully integrated with my website. I have a couple of other plugins to automate my video management for me, but otherwise, it’s all good to go.
But wait, there’s work…lots and lots of work
Yep, it is work. Work that I’ve done before, in a sense, but I can re-use that work from before. Captions, album descriptions, consistent workflows, etc., it’s all saved on my site. So much so, that I have it nailed as a twelve-step process to get a gallery (what I used to call albums) up and running (a single month is a gallery, for example). Here is the process:
SORT THE PHOTOS — This is MOSTLY already done. I have a good file structure that distinguishes between “extra” photos and what I consider “production” photos, i.e. the ones that I’m willing to share. Sometimes that might be 10 group photos where only one has everyone looking the right way. The other nine go in a sub-folder called EXTRAS, the good one goes in a root. But I do have a bit of tweaking here and there to do for the files, such as breaking really large galleries into 2 or 3 by event rather than just dumping the whole month in a single folder. Sometimes that is either a special trip during the month to, say, Toronto or Montreal; in another, I have 6 folders of day to day life doing various things, and 1 folder of a wedding with 100 photos of family. If I dump them all in one MONTHLY folder, it gets unwieldy to navigate. Not impossible, but a few times when I was working with the old photo galleries, I thought, “Hmm, maybe I should have organized that differently.” Now, since I’m “redoing” some of it anyway, I can fix it as I go. AKA the “anal retentive” step.
STAGING — Before I import into Mylio, I like to make a separate copy of just the production photos and put them in a separate folder. Then my import is completely clean with no chance of huge duplicates. Nor do I end up with the videos clogging the sub-system. This also has an extra advantage I hadn’t foreseen — when I go to make photo books later, as I want to do, they will already be “reduced” down to the key ones to consider.
INTO MYLIO — While this is generally a question of just importing, I also do the facial recognition at the same time. I have it scan all the photos, do the best job it can in finding faces, and then it prompts me to identify people in batches. If you think of a wedding as a good example, there will likely be a fair number of photos of the bride. Almost always in good light facing the camera. At least in theory. So Mylio is going to be able to tell her across a bunch of photos in each folder. Then it asks me, “Who’s this?” and groups every face that seems to match that same configuration i.e. all the faces of the bride look the same, and has me answer. “Jill” for example. Then it tags them all with Jane’s name. Then it shows me face group 2 — likely a slightly smaller subset of some dude’s face, in a hetero couple at least — and voila, I can tag “Jack” for all the photos that have a face that matches Jack’s in it. And it shows me the set that it thinks are all Jack so I can quickly verify before tagging them all. Oh wait, it got a slightly blurry face in there too that it thought was Jack but is really his cousin Bob. Tag that one out, tag the rest as Bob. And so on. Until it gets down to a very small set of photos that it doesn’t know who they are from the database, and not that many to group. So it gives you a photo, maybe one that has Jack and Jill already tagged, along with Aunt Martha. So now you add the tag for Martha. How well does it work? Pretty impressively actually and pretty funnily too at times. I have tagged myself at age 20 and 14, and bam, it looked at a photo of my family where I’m under age 5, and it said, “Hey, is this Paul?”. Yes, it makes predictions. On the funny side, it looked at another photo of my grandmother and thought it was my brother Mike. Another one was a photo of my dad holding a garden gnome, and the computer thought the gnome was a person — my sister Marie. I REALLY wanted to click, “Why, yes, my sister IS a garden gnome” but that seemed counter-productive for a reliable database. Instead, I can just click ignore on faces that are not actually people or even ignore faces in photos where there are 4 strangers in a street scene.
MYLIO FOLDERS — The import feature is a bit tempermental, partly as I want to make sure the folder names in Mylio are consistent. So I tend to import them, and then play with them a bit. Nothing major, just some minor cleaning up in a working sub-area and then “moving” them where they should go.
MYLIO KEYWORDS — I then tag a group of photos, and add some keywords. Could be simple like “Wedding, Family, Cottage” for cousins who got married at the cottage. Or could be “Trip, travel, Bangladesh” for a trip Andrea took to Bangladesh. Any photo that has identified / tagged people in gets their names added to the keywords too as these show up as “tags” in WordPress later. The benefit of that is that when everything is uploaded, I can show a “tag cloud” and click on the tags to see a virtual album of all the photos that match. Such as “Jupiter” for all my astro photos of Jupiter. Or “planets”.
MYLIO CAPTIONS — While keywords handle the themes, I usually like to have captions for each of the photos that describes more specifically what is in the photo. It often is a series of captions for sub-groups of photos. Like “dancing” for a bunch of photos from the wedding. Or “skating on the canal” for four or five shots that are all about skating on the canal. However, sometimes there will be a sub-shot that I’ll be more specific about like “first dance” while the rest are just “dancing at the wedding”. I’ve often grouped photos that way in my previous galleries rather than trying to name each photo separately. It’s hard to be creative enough to say “dancing” 20 times in different ways.
NEXT GEN GALLERY UPLOAD — Okay, finally, I’ve got a gallery in Mylio ready to go, and I have to upload it to WordPress. I could “cheat” and try to import the photos directly from the Piwigo site a few cyber folders away, but while it would save upload time, I’d lose all the captions and keywords. Hence why I’m doing it completely anally this time. I name the new gallery according to a very specific filenaming convention to give me an easily sortable list, add file links, and say UPLOAD. Oddly, it has a limited number of formats that it likes. Video is not one of them, and if it hits a video, it just ignores it. I would rather it said, “Hey, VIDEO HERE”. But alas, it doesn’t.
CREATE A GALLERY PAGE — Now that I’ve uploaded a gallery, such as 2005-01 January, I create a page on the website to “embed” the gallery with the right layout, templates, etc. and ensuring the right file structure so the URLs have a simple and easy to navigate structure. And making sure it doesn’t conflict with my PIWIGO install. Yikes. I also add a light touch to the page with a description describing the photos in the gallery (mostly a copy/paste from earlier descriptions). Unfortunately, Mylio doesn’t do descriptions of folders, just images. So this part is still manual.
ADD VIDEOS — Before finishingthe page, I upload any videos for that gallery and save them manually to the bottom of the Thumbnail page. Add a few captions, tweak the layout a bit for size of the video player, then save, publish, and the page is good to go.
MODIFY GALLERY DESCRIPTION — Once I have written my little description as the intro to the page, I copy and paste it over to the actual database and have Next Gen Gallery remember it. There’s an option to display ALBUMS of sub-galleries, and I can have it both keep a description to show online in a virtual page, as well as make sure cover photos are showing for each gallery.
FACEBOOK — Once the gallery is “finished” and the page is good to go, I share it on Facebook, along with the description and tag Andrea.
MOVE BACKUP PICTURES — Back at the start, I moved a temporary copy into a working folder for Mylio to play with. The import process creates a copy and saves it somewhere else, so I can delete the folder. However, I do have two other folders to manage — the collection of photos that have been successfully uploaded, and the collection of photos still to be processed. Both include ALL the photos — extras, videos, maybe even some text files if there was something relevant to the “event”. All those get moved to a backup directory so that I don’t lose everything if I lose a disk drive in my computer. Plus of course all the areas (Mylio, TO BE UPLOADED, and UPLOADED) are backed up separately too.
And since I’m completely anal, I have a tracking sheet for each year. For 2005, I have 19 separate galleries to do (12 months plus 7 special events) with 12 steps above. This means a total of 228 steps to cover the year. Some are a bit time-consuming, some are pretty short. But I track them to make sure I don’t miss any steps and suddenly find myself with “some photos” processed and others in the same folders not even reviewed. Like I said, anal-retentive.
I wouldn’t say I’m completely satisfied with everything, but I do really like having it all in one install of WordPress rather than separate installs. Particularly as I often blog about events that match the photos — now I can embed the gallery and blog around the pics more easily than embedding separate photos as I had to before.
It’s a work in progress, as my blog always is. But I’m pretty satisfied with my progress so far. Ask me in a year when I get through all the galleries and get caught back up.
Phase III: Fixing an embedding problem in WordPress
One of the things I do with my photos, besides having them in a gallery, is embed them in various posts in my WordPress blog. For example, I have a section on the site dealing with an HR student conference from back in 2002, and I have a small album of photos with the conference docs. Those photos are stored with my Piwigo gallery, and embedded as a hot link in the WordPress pages. Simple, right?. But here’s the problem. The link to each photo currently says:
http://thepandafamily.smugmug.com/yada yada yada
Now that I have the new gallery up and running, if I simply delete the old one, those links won’t work. I have to change ALL of them to say:
http://www.polywogg.ca/pandafamily/yada yada yada
It isn’t a huge challenge, just under 100 posts in total with maybe 400 photos linked. But each photo or video link has to have the SmugMug link deleted and the Piwigo link pasted BEFORE I delete the SmugMug account. If I don’t do it first, then my WordPress site will suddenly have a bunch of broken links all through it and no photos showing from my gallery.
But of course it isn’t as simple as just a search and replace of the opening domain info — the “yada yada yada” is completely different for each site. So they have to be done manually. Since it is easier to do while the two galleries are both running, i.e. so I can view them side-by-side on the screen and copy the links from the new to replace the old, it is still a pain in the patootie. With the uploading and captioning done, I’m about 50% done the re-linking process. But I got my account renewal reminder the other day … SmugMug renews in less than a month and I wanted to be done before then so I won’t get charged for another year.
I did the first few, and they were easy-peasy. So I thought the “rest” would be the same. Strange, but I feel like it was both less work and more work than I expected. How can that be?
Well, I feel like there were “only 98” posts with the cross-linked photos, which seemed like a manageable number. In addition, many of them only had one or two photos, so pretty quick. All in all, that meant I was initially feeling like it was less work than I expected and would go pretty fast.
Right up until I hit some of the photo-rich posts like stories about Being Jacob’s father or various trips we took. Some of them took a LONG time to update. But the weird part is I feel like the photos are somehow “brighter”? That’s weird. I wonder if the filters and themes at Smugmug that I was using were muted somehow. Anyway, I really like how it looks now.
And I’m finally done. It took a bit of time, maybe 6 or 7 hours in total to do the updating of the 98 posts, although in fairness, some of that was because I was sucked into reading my own posts again and editing a bit as I went. 🙂
I was right, uploading took a whack of time. I also don’t much like one aspect of the upload window — if something “fails”, it gives you an error while the screen is still uploading so you can see it, but once the rest of the uploads are done, it just rolls over to a new screen showing the successful uploads. No continued error message to say “64 uploaded, 2 didn’t”. So I wasn’t monitoring as I went, and later in the subsequent phases, I’ve discovered “missing” photos and videos i.e. ones that for some reason didn’t upload successfully the first time. Not a huge problem to fix, just annoying. If the result page showed the “failed” ones, I would have fixed immediately upon upload.
I also underestimated the final size. I thought about 10K in photos, which is about right for the family photos. But with everything else on the site, there are actually 14,147 photos, 500 albums, 24 plugins and 25.1 GB of data. Wow. But that part is “done” (small caveat — there are a few months where there are some special photo collections my wife took, so I’ll need her to figure out which ones of those should be included).
Phase II: Preparing the folders and pictures for viewing
While uploading took time, it was generally mindless, something I could spend a few minutes sorting and adding in the ones to upload, and then clicking the button to start. It could take 60 seconds, or 10 minutes, depending on how many pics or how long of videos, but it was background computer stuff while I do other things.
But once the upload was complete, I also had to start playing with the files and albums online to make them presentable. Oddly enough, one of the first things I had to do is tell it to generate all the “little” thumbnail and square size photos in the background. It does it fast enough, the server I mean, while I wait. Another background task. But I need it done because the second step is to play with the display order, and while doing that, I need to be able to move files around by looking at their little thumbnails. But once uploaded, it’s ON the website, not within a file browser, so there’s no “viewing processor” running to let me see it easily. Instead, the website creates the little thumbnails as extra files and then displays them for manipulation.
In an easier world, the photos I was uploading would all have the exact same filenaming taxonomy, and thus once uploaded, I could sort by the creation date (for example) and everything would be in order.
Except some of the pics come from my DSLR. Others come from Andrea’s iPhone. Others come from a small pocket camera. And still others come from two different apps in my Android phone. Which means they all have their own filenaming convention, and they don’t “sort” easily. And if I edited them at all on the computer, with a crop for example, often the software changes the metainfo so that the file creation date is the date I did the editing, not the original “taken” date. Don’t even get me started on images sent to me by other people where they’ve named them “Dave and Janet at the lake” and then “At the lake with Dave and Janet”. The anal-retentive side of me wanted to impose a filenaming convention, sort them all, get them looking identical, and then upload.
But that is way overkill when it takes me 60 seconds of viewing on my desktop to decide on which photo I’m looking for in a batch. This isn’t a “shared” server where we all have to use the same convention. Ultimately I don’t care what the filename is, other than for quick reference. But, since I can’t rely on the filename or the creation date, I do a manual sort. Most of the time, I do a default filename sort plus the original upload order, and then I just move a few things around. Like putting all the photos of Uncle Dave together, even if I took a couple of other people in between.
However, merely putting them in a good order is not necessarily the biggest job. In most cases, since I already had a working gallery elsewhere, with the same photos already uploaded there (alas, I couldn’t transfer directly), most of the time I’m just matching the new gallery’s order to the old gallery’s order. So, again, most of the time, the order isn’t that time consuming. But for some reason, one of the ones I did today was brutal (about 75 photos in the middle of the batch didn’t get uploaded, and when I did upload them, it gave me a huge batch at the end of the collection that had to be moved — one by one — up to the right space).
At this point, I had a gallery with pictures and videos in them, sounds good, right? Except they had no captions. I mentioned in an earlier post that I was annoyed that I had to put the same info twice in the meta data — once for title so it would appear on the album page, and once for description so it would appear on the single photo pages. I reached out to the Piwigo community, and heard nothing back over the course of a week or two. Okay, I guessed I would have to paste it twice. Then it occurred to me. I had chosen a theme where I *should* be able to alter this in the template, but in reading the template files, I couldn’t find the fields to change. I was looking for something called TITLE or NAME and DESCRIPTION, since that is what the admin pages call them. So I posted on the discussion page for my particular theme, hoping successfully that the creator of the theme would respond.
Which he did. Except his first response was “Good idea, make it a plugin and upload it to the repository”. Except if I *could* do that level of techno programming, I would have already done it. I couldn’t even FIND the fields to work with. So I went looking again, and found two rows of code that looked promising and I posted an update to my question.
So I found picture.tpl and the refs to description include:
data-description=”{$thumbnail.DESCRIPTION}”
in two places. I could change that to $thumbnail.NAME. That would be telling it that the description never gets displayed, I think, just that the field will be the name/title field. It also exists in index.TPL.
Although perhaps I’d be better off trying it as “data-description=”{$thumbnail.NAME}” & “{$thumbnail.DESCRIPTION}” ??
My thought was either to change the template to always show just the TITLE field in both album and picture pages, OR to do a little replacement code to tell it that when it went to display the description, to just first copy the text from the TITLE into it. So either show the title or copy the title into the description and then show the description. Either way, the title would show. Or so I thought. Turns out I was TOTALLY off-base.
The text below the main image is set in [Github] piwigo-bootstrap-darkroom file
template/picture.tpl@L38
No idea why the variable is called $COMMENT_IMG, but it’s the description. If you replace the two $COMMENT_IMGs with $current.TITLE it should do what you want for now.
The data-description stuff is for the PhotoSwipe slideshow.
Of course. The title / description is called COMMENT. Which are not to be confused with the actual comment fields. While the TITLE field means something else. Why didn’t I figure that out on my own? 🙂
Who cares in the end? Not me, cuz I made the tweaks and damned if it didn’t work EXACTLY the way I wanted it to do. Fan-freaking- tastic! No more entering the captions twice. Whew!
Now I still had to set captions for about 10,000 photos, and while some of those were done in batches (i.e. multiple photos with the exact same caption like “Small deer at Parc Omega”), others were variations on a theme (“Day 06 – Trip to Cozumel – Water park” or “Day 06 – Trip to Cozumel – Lighthouse”). Others were individual. A fair amount of work.
The last two things I had to do before each album was ready was to test all the photos and videos to make sure they display or play properly (once in a while, a video wouldn’t play, or I had audio but no video, or the picture was upside down), and then, when all was ready, choose an image from the batch to serve as the image for the album cover.
Generate thumbnails, sort the photos, fix the captions, test the viewing, and choose a cover image. It went a lot faster than I initially thought, but it could not be done quickly or in the background. I had some decent processes in place for a good workflow, but it still required me to do a lot of the grunt work manually.
I finally finished after about two months of work, doing a few albums at a time.
