Tag Archives: photos

Figuring out my FULL workflow for my photo gallery

The PolyBlog
January 11 2021

I mentioned in my posts about my goals for 2021 that I want to do a fair amount of work on my photo gallery (Setting goals for 2021 – Part 5: Computers, Website, Blogging, Writing, Media and Photos). But in order to do that, one of the things that is a challenge is ensuring that each gallery (say, a given month of a year) is set up consistently each and every time. That’s not a “small” step of consistency, but one that starts from the very beginning. So let’s talk about the different stages of the workflow.

Stage 1. Taking the photos

My photo gallery is made up of photos that come to me from multiple sources, and I need to be able to tell them apart for the purposes of managing. I can manipulate the filenames after the fact, but it would be easier (as my friend Matt suggested) if the filenames themselves were a bit more process-friendly. So let’s look at what those FNs would look like:

  • Andrea’s iPhone –> IMGxxxx – YYYYMMDD – AH
  • Paul’s iPhone –> IMGxxxx – YYYYMMDD – PS
  • Jacob’s iPhone –> IMGxxxx – YYYYMMDD – JH
  • DSLR –> IMGxxxx – YYYYMMDD – DSLR
  • Point and Shoot –> IMGxxxx – YYYYMMDD – SC (* for small camera)
  • Screen grab –> IMGxxxx – YYYYMMDD – SCR
  • Tablet –> IMGxxxx – YYYYMMDD – TAB

Now, some of those can be assigned at time of capture, while others will have to be assigned at time of file transfer. Depends on the original tool. So the workflow bifurcates after this step, but the step here is the same:

  • Take the picture

Stage 2. Managing the photos – Pre-processing and pre-sorting

Managing the photos is initially quite easy. I have to copy them from the device to the same folder on my PC, and I do it in four stages depending on the imaging source (copy to the hard drive, copy to the sorting folder, simple sort, and convert videos).

For Andrea’s iPhone and the Point and Shoot camera, the photos get transferred to her desktop and then copied onto a USB thumb drive.

For Jacob’s iPhone, the photos get transferred to his laptop and then copied onto a USB thumb drive.

For my iPhone, the DSLR, and the tablet, all the files are synched onto my desktop into a Synching folder.

Any files that are not already renamed properly can be renamed at this stage.

For the final step, all eight original sources are then moved into a sorting folder where I sort them by month and day or event into relevant folders. Since lots of shots are grouped together, I create a file folder structure such as:

YYYY / MM-Month (general folder as the upper level folder)

YYYY / MM-Month / DD (Event title) like “14 Kayaking at MEP’s” or “29 Birthday party”

YYYY / MM-Month / Extras as a folder for ones not being used in the gallery (often mistakes or just not meant for public, like a photo taken in a store of something I want to buy)

YYYY / MMb – Special – Special event for the month (#1) like a party, trip, etc. where a large number of photos warrant their own gallery

YYYY / MMb – Special – Special event for the month (#2) like a party, trip, etc. where a large number of photos warrant their own gallery

YYYY / MMz – Blog posts for photos that I’m going to eventually upload to the website to include in various blog posts but are generally not interesting enough on their own to include in “general photos”.

For the first level of sorting, I move everything into the relevant folders. It may mean, for example, that I have a folder for a big trip that has photos from multiple sources in it, and in fact, I usually do have at least 2 sources for various daily events.

For the second level of sorting, I convert all MOV format videos into MP4 format (suitable for the web) and move all old videos into the EXTRAs sub-folder.

  • Copy from the device to the hard drive
  • Copy from hard drive (potentially via USB flash drive) to the sorting folder and rename any if necessary.
  • Do a simple sort by event and dates
  • Convert videos to MP4

Stage 3. Managing the photos – Advanced sort in Mylio

I use Mylio as my image manager, and I do four steps in Mylio.

First and foremost, I import all the images from the sorting folder including the directories I created. When they arrive in Mylio, they are in a sorting folder too. I basically go through and move the quality images I want to use for each event into the MONTHLY folder (such as 2021 / 01 January). Extras that I’m not using, such as the secondary or tertiary photos of a group of ducks, I move into the EXTRAS folder. I also do a quality sort on the videos, special events, and folders of photos I intend to use for my blog posts.

In some cases, I may decide to edit a video or photo to make it suitable for sharing, in which case I make a copy and edit the copy rather than the original. I usually do this in another program beyond Mylio and then reimport the edited version.

Once I have a set of photos and videos for a given month (for instance, 2021 / 01 January), I run facial recognition on the “good” photos (there are too many photos to worry about doing the Extras too) and I let the computer do most of the work to group them and guesstimate who is in the photo. After the first few hundred of a given year are posted, the rest of the guesses are usually pretty accurate on the first attempt. I then add metadata to the files. This includes a name and description for the photo (identical as it is used differently in WordPress), something short, and some keyword tagging that includes year, city, event, and any people in the photo who were tagged in the Key Words. Finally, I save all the metadata to the image file.

Finally, now that the photos are all sorted, named and tagged, I move it from a sorting folder into my full folder structure for the Panda Photo Gallery in Mylio which generally has the structure of FAMILY / YEAR / MONTH.

  • Import images into Mylio sorting area
  • Do advanced quality and photo selection, filing the rest in EXTRAS.
  • (Optional) Edit any photos or videos that require tweaking
  • Facial recognition
  • Add metadata (name, description, tags/keywords)
  • Save metadata to image file
  • Move from sorting folder into a full folder structure

Stage 4. Uploading to WordPress

Up until this point, most of the file management stuff is just simply a good process / workflow for keeping my photos organized and filtered for quality. Now I look at the parts of getting it on the website.

Initially, I create a page to hold the gallery (while this could be done later, it saves a step in the gallery creation process) and assign it a name such as 2021/01 January, and insert two default items — a blank photo gallery and a blank video gallery. I save the page, but leave it in edit mode.

Then, working by folder (such as 2021 / 01 January), I create a new GALLERY called 2021-01 January in WordPress. This opens an upload area, and using Mylio as my initial interface, I upload all of the good PHOTOS for that month/event. Since I already populated the metadata fields, the upload puts everything into the WordPress fields for me. This completes the step of uploading all the PHOTOs. I can then edit the GALLERY description to describe the various events in the monthly folder (copying the description for later use, like a descriptive table of contents), add the link to the page created above (the step that I saved allows me to do this now), and add the GALLERY to the appropriate ALBUM (such as the year, 2021).

Then it is time to upload Videos, if any. These have to be done in the Media Library, and using Media Library Assistant, I save them to a separate sub-folder usually called YYYY-MM. Now that all the files are uploaded, it’s time to go back and edit the page I created earlier.

Each Gallery page has six components to edit:

  • The Page name, if it needs to be tweaked from the standard page name (usually YYYY-MM for months but could be YYYY-MM Special – Trip to Mexico);
  • A manual breadcrumb that I’ve created to allow the viewer to go one level higher easily;
  • A description of the gallery (same as what was already entered in the Gallery Page, just pasted here);
  • The blank gallery block to choose which gallery I want to show, and to change the order of photos if needed;
  • The names / description of any videos that need to be linked; and,
  • Linking to the videos themselves.

Finally, everything is saved and the page is previewed to make sure everything works, and the page is published. The link is then shared to FB along with the gallery description.

  • Create a page and edit the page name, add a blank photo gallery block and a blank video gallery block, and save in draft mode;
  • Create the new gallery, upload all the photos, edit the gallery for gallery description (and copy for later) and link to the page created above;
  • Add the GALLERY to the right ALBUM;
  • Upload videos and sort into sub-folder;
  • Go back and edit the page for page name, manual breadcrumb, paste the description, choose a gallery in the blank gallery block, edit the names / descriptions of the videos, and link to the videos themselves;
  • Save and preview/test, then publish;
  • Share link with FB.

Stage 5. Backups and further usage

Mylio automatically does a backup of all photos to a secondary location, and long-term, I want that to upload to the cloud too. Later, I do a separate backup of all my files to off-site storage.

At the end of the year, I also take all the “GOOD” photos and put them on a USB thumb drive for Andrea to weed and use to make a Photobook. Once she’s done, I save the final photos back to another folder labelled PHOTOBOOK. If there are any really good ones for the year, we also use them in Calendars, New Year’s letters, metal prints, and an e-frame.

  • Backup to secondary location/vault;
  • Backup with all files to offsite location;
  • Create a small subset each year for Andrea to use for photobooks, calendars, New Year’s letter, metal prints, and e-frame;
  • Copy subset back to a folder called PHOTOBOOKS.

And then, finally, I’m done. Whew. So let’s look at that workflow all together so I don’t miss anything each time. I’m also going to copy it into a PowerPoint print-out so I don’t lose it. Nineteen steps that I have to do consistently every time or something gets messed up.

  1. Take the picture
  2. Copy from the device to the hard drive
  3. Copy from hard drive (potentially via USB flash drive) to the sorting folder and rename any if necessary.
  4. Do a simple sort by event and dates
  5. Convert videos to MP4
  6. Import images into Mylio sorting area
  7. Do advanced quality and photo selection, filing the rest in EXTRAS.
  8. (Optional) Edit any photos or videos that require tweaking
  9. Facial recognition
  10. Add metadata (name, description, tags/keywords)
  11. Save metadata to image file
  12. Move from sorting folder into a full folder structure
  13. Create a page and edit the page name, add a blank photo gallery block and a blank video gallery block, and save in draft mode;
  14. Create the new gallery, upload all the photos, edit the gallery for gallery description (and copy for later) and link to the page created above;
  15. Add the GALLERY to the right ALBUM;
  16. Upload videos and sort into sub-folder;
  17. Go back and edit the page for page name, manual breadcrumb, paste the description, choose a gallery in the blank gallery block, edit the names / descriptions of the videos, and link to the videos themselves;
  18. Save and preview/test, then publish;
  19. Share link with FB.
  20. Backup to secondary location/vault;
  21. Backup with all files to offsite location;
  22. Create a small subset each year for Andrea to use for photobooks, calendars, New Year’s letter, metal prints, and e-frame;
  23. Copy subset back to a folder called PHOTOBOOKS.

Finding free images from the internet for my blog

The PolyBlog
April 16 2020

Many people search on Google and steal any images they find willy-nilly without ever considering the copyright on the original photos. Mostly people who think everything on the net is public domain. But if you’re running a proper blog, and creating your own content, you know that’s not true. But that leaves you with three choices for photos and graphics:

  • Make your own (on your own desktop or using apps and websites);
  • Buy it from some pay-per-photo sites; or,
  • Find free sites.

Lots of sites advertise “free” photos but many are the internet’s version of a drug dealer…they’ll offer you a free sample of a couple of photos or graphics to give you a taste, and once you’re hooked, charge you through the nose for the next batch. And the one after that. And suggest you upgrade to a subscription model which seems incredibly cheap when you’re first setting up your blog when you need dozens of photos, but if you don’t use tons of photos in your site later, it’s a bit of a cash cow for them.

However, earlier this week, someone posted the question on the WP Beginner group on FaceBook, asking where to find images. And the responses poured in rapidly. Some are good, some were silly. Here are the ones I found worthwhile.

I should flag that while many of the other suggested sites had decent photos, many had licenses that were restrictive. For example, many require attribution to say “Image by Jane Photographer”. Which is fair. But if I’m using it for a featured image on my blog, which might be attached to multiple pages or posts, how do you include that? Some people put it in the ALT TEXT, but that isn’t visible and doesn’t meet the requirements of the license. Others just add it to the footer, or create a credits page. All of which are doable, but not something I am ready to add to my updating regime. No, I want free for personal or commercial use (my site isn’t commercial, but some jurisdictions might treat it as such), no attribution required. Often you can find sites that require either attribution or a paid membership, but for a random photo here and there? Not really worth it to me. I already have my own photos as well as extensive clipart collections I’ve bought over the years. If I have to pay again, I can probably find an alternate image.

Photo sites

The most popular one for photos is called Unsplash. I’ve blogged about it before, and at the time, my reaction was:

A site called Unsplash has free images for use, and many of them are even available for commercial use if need be. People, landscape, buildings…lots of choices. One of the things I often “test” when I’m looking at a site is whether or not they have photos of frogs and what kinds. Simple or full-on tree frogs. It’s what I want for PolyWogg from time to time, so it’s a “real” test. Unsplash has a bunch […] which isn’t bad, and while none of them are OMG AWESOME, they’re certainly usable for a blog post if I want something. For those doing an article about a city, you’re likely to find well-known landmarks easily.

That was written a little over a year ago, and so I played with it a bit more today. I’m quite impressed with the sheer volume. And the updated license is pretty unequivocal:

All photos published on Unsplash can be used for free. You can use them for commercial and noncommercial purposes. You do not need to ask permission from or provide credit to the photographer or Unsplash, although it is appreciated when possible.

https://unsplash.com/license | Unsplash

Another popular suggestion is Pexels.com. It too has an open license, although if you want to use the images on commercial products, you have to modify it somehow (no straight inclusion). However, they also have video options to include, not that I can think of when I would ever want that function. Stock footage, I guess. Pixabay is also popular, and includes a few graphics here and there (* note my revision below). Not many, and the randomness seems almost accidental. Their license is similar to Pexels…if you want to print it on something commercial, you have to adapt it first. Another backup option suggested was Foter. Decent, not inspiring.

Graphics, vectors and clipart

But honestly, outside of some meme creation where I might use photos in the background, my main need for my site is not photos but graphics or clipart (the latest “phrasing” is to call them all “vectors”). For featured images in my blog, I usually prefer graphics, even whimsical ones, to actual photos, so I went looking for good sites.

So I too asked on WP Beginner. Oddly enough, Pixabay came up again as a suggestion. I had noted above that when I searched, there were some graphics with it, but it seemed random/accidental. I really only saw photos. However, when I saw Pixabay come up again as a suggestion but for graphics, I went looking again. Sure enough, there is a “vectors” area. The default search says it will search all of the areas at once, but I didn’t find those options earlier, so I must have somehow had a filter on. I found a really great graphic that might be a better option than another 5 or 6 that I already had as options. Nice!

I also found Public Domain Vectors which has a great collection. Lots of choice, lots of options. Most likely my goto option with Pixabay.

Now I’m ready to rock.

Upgrading some features on my website…

The PolyBlog
March 21 2020

I’m sure my wife saw the post title and started social distancing just for that. “Not again!” was likely her thought. It’s true, I do play with some stuff on the site, often figuring out new ways to do something, and since I’m anal-retentive, I hate the thought of something that leaving previous versions if, say, I find a better way to do book reviews that I would implement starting now.

Simple content areas

Most of my content is relatively straightforward — a blog post here, a blog post there. For each, they are pretty text-heavy even if the popular website wisdom is more graphics and video. That’s not me, I’m a writer, I write words. But there are a few areas where I feel the choices for how to display the text are not quite so clear; for the simple areas, it is relatively, umm, simple.

For astronomy, I share my own pics of course, but I’m also writing an astronomy guide. So having a simplified layout that is easy for anyone to read is important to me. Mostly so far it is only a table of contents and a series of early pages or blog posts. I can do them as either (post or page), really, but most are done as pages.

For my challenges (reading, baking/cooking), they’re relatively simple static pages.

For materials related to government, much of it is simply one-off posts, no real structure required. But then I have two other areas…the first is PS Transitions FP, a report from a conference that a group of students from Carleton organized in 2002, and for which I was the webmaster. It’s entirely static, but it does have some tabbing in it, as well as a photo gallery. I’ve kept the content there for over 18 years, but the methodology for doing so has had to be altered a little bit when plugins expired, or setups on my site changed.

The other area is my HR guide, and it has been a challenge more for organizing than content, at least in terms of the website portion. I have multiple versions of some of the content, with a LOT of comments on pages that I now consider archival. I hesitate to delete them and lose all those comments, but I don’t like having the old versions of the guide there when I’ve written later and better versions. I recently found a plugin that will let me move comments from an old post to my latest post on the same topic, and I’ll likely consolidate it all when I get my latest version of the guide finished. My wife is acting as my editor, so I’m hopeful it will be my best version yet. And then I’ll likely delete all the other content. The thought makes me queasy, to be honest. All those words, used extensively by people, but I’m going to delete entire posts and pages? I haven’t worked through that mentally yet. I might find a way to preserve it somewhere else on the site.

Under personal, I have posts about family and goals, all relatively straight-forward. But the ones for humour and quotes give me pause. I like the idea of sharing both through social media as memes. And then including them on my website. Sounds simple, right? Except if I do it as a meme, i.e. a graphic, then the graphic doesn’t get indexed on my site. Index bots don’t read the “text” within the graphic, it is just the graphic. So if I add a long joke, or even a short quote, and someone was to search anywhere for it, my site wouldn’t show up in the search engine because technically that text doesn’t appear on my site. Yet, by the same token, if I post it as text, it doesn’t look as sharp as a meme, suitable for sharing. Someone suggested including both, but that seems redundant. However, I might have a new way to at least create a searchable list of the description of the meme at least. A bit manual to create at first, but ongoing would be simple to update for future posts.

The more difficult areas to format

The real challenge has always been my reviews. Before I even had a website, I wrote book and movie reviews and just shared them along with jokes to a subscription-based newsletter list. It was free, but you had to “ask” to be on it. I had book reviews, movie reviews, jokes, and an active trivia game at the time. Most of it was in a spreadsheet that handled all my formatting in ASCII format so I could paste into an email and just pressed SEND. It worked, I liked it, and when I created my first PolyWogg website, I wanted to put my reviews there.

Of late, though, I have mostly focused on book reviews. I generally have liked the format for them (plot / premise, what I liked, what I didn’t like, bottom line, rating), and yet I confess it took me several tries to get them looking the way I wanted. One of the early challenges was whether or not I include a “disclosure” phrase in the review on my own website. You’re supposed to declare any conflict-of-interest elements if you post reviews on a lot of commercial sites, and since I share them on those sites too, it seemed simple to also include them on mine. But over time, I realized I didn’t really care. For almost none of them do I have any conflict. I don’t have that many cases where I got a free book / advanced reader copy to read. So the disclosure was bulky and just said that I had no link to the author. Kind of meaningless in the long-run. I cut it.

Then Amazon started playing with how they handle referral links. I didn’t have a lot of links on my site, and so I wasn’t getting any referral money. I think I got about a dollar over five years. But I did have the account, and the main point of having it was so I could link to the Amazon website and hotlink in pictures of the book covers. Yet Amazon booted me from the active referral program along with 1000s of other affiliates who hadn’t earned any commissions in the previous year. They culled the list, so to speak, and I would have cut me too. But that called into question my hotlinking, which also required me to run two extra plugins. Could I get the images some other way? Yep, Good Reads grants permission to those doing reviews to link to the images on their own site. Yes! So I went through probably 150 or so book reviews, reformatting a few things as I went (like cutting disclosure paragraphs) and updating all the images. Tedious, but they were all “fixed” to match the new approach. That was about six months ago.

The part that was “left” was my index of book reviews. I had tried some indexing tools, some table plugins, a few other things, and none of them really worked the way I wanted them to do. Because I had different types of info that I wanted to be able to group by:

  1. Alphabetically by title (obvious);
  2. Alphabetically by author;
  3. The raw review number (i.e., mostly chronologically for me for the order in which I write the reviews);
  4. The date of my review (where #3 failed is some of my reviews are old and I’ve updated them and included them, but that means putting in a 1998 review in between two 2015 reviews, for example, so #3 and #4 are not exactly the same sort);
  5. The year the book was published;
  6. Series and order, to give me the ability to group books in the same series; and,
  7. My rating.

But without the proper tool to display all of that, I organize it manually. I still use a flat-file database in a spreadsheet, Excel currently, although it started off in Lotus 1-2-3 years ago, and in the spreadsheet, I have a field that formats the info so I can simply paste it into my website. For example, I mix and match sub-fields into a single string that says:

TITLE by AUTHOR (BR#####; published {date}; reviewed {date}; series {order}; rating)

I then simply paste that into one page, add a hotlink to the URL for the review post, and then copy that to six other index pages. I have tended to do it in batches of twenty-five book reviews at a time, so I would write 25 reviews and post them over the course of a number of months, and when I got to the 25th, I would then paste all 25 strings into a web-page, add the URLs, and then paste them into the other six pages too. Time-consuming, and doing 25 together made it a bit more efficient on workflows, but it was a workflow blockage too. Plus, once in awhile, I’d mess up some link or a copy and paste, and then a year later, I would happen to notice that the link from one of the indices was not, in fact, linking to the right page. I’d messed it up, and when I beta-tested it, I had apparently missed the errors. REALLY annoying. Another downside to coding some things manually.

As I said, though, I had tried out a bunch of options to put it into various auto-sorting tools, but it never worked well.

An accidental revolution

In addition to my book reviews, I also do movie reviews, music reviews (although mainly only one year so far), and TV reviews. For the TV reviews, it is INCREDIBLY slow to do a review of a full season of TV for me. Which is odd, because the individual episodes are ALREADY reviewed. As I watch TV, I keep track of individual episodes and when I finish the episode, I use a similar spreadsheet to automate a quick TWEET that says:

ShowTitle – S(eason)##E(pisode)## – EpisodeTitle – QuickOneLineReview – RatingOutOfFive

Sooo, I have always wanted to embed those reviews in my website, but didn’t have a good way to do that, at least not quickly. I tried a manual approach:

  • Created a table in a reusable post template;
  • Added a line for the Tweet;
  • Added a line for a picture from the episode (I was saving them for a while);
  • Added some areas to talk about the overall season;
  • Added an area to rate the whole season;

But then I was stuck. That is a LOT of copy and pasting to get it to look right. I tried just pasting from Excel spreadsheets, but the paste is painful — it adds codes to EVERY cell, so if you want to adjust layout later, the whole table is a mess of codes. So I went looking for a way to embed an Excel Spreadsheet into a website easily. Just so I could paste, for example, a whole season’s worth of tweet/reviews at once.

And I found the very popular plugin for WordPress called TablePress. It would allow me to import spreadsheets directly or even to paste them raw. Gave it a try, and BAM, it worked right out of the box. Great, I had a way to paste the whole season at once into a page.

But then I noticed some other features. It would let me search the table too, applying the terms like a filter. Not really needed in a table of only 20-25 rows, but interesting. Oh, and you can sort columns too. Again, I don’t really need that with the episodes.

Or more accurately, I don’t need that function for THIS table. But what about my book review indices? Holy Hannah, I could have ONE table instead of 7 and EVERY FIELD is sortable? Plus I can paste directly from Excel? Holy fudgicles!

Welcome to the revolution

I only had 8 reviews of TV seasons, all for the show Castle that finished a few years ago. Again, as I said, too time-consuming to paste in every episode line by line, particularly if I was also pasting in photos. Meh. Instead, I’ve cut it down to an overview, episodes that I liked, those that were watchable, those that I didn’t like, a table of all the episodes, an overall review of the season, and some links to the index of other reviews.

With each column sortable. I copy the rows and columns from Excel where I already have the info, paste it all into the back end, add one line to my page, and BAM!, instant table. I started thinking, okay, this is good, I’ll do a table for each season, no problem. But then I thought again. Every table will be identical in format. And Castle has 8 seasons, that’s 8 tables to keep in the database with different names, I’ll need a good naming convention, etc. Hmm…but what if I could merge ALL of the Castle episodes into one table and just list those that correspond to Season 1. Is that doable? Turns out it is. TablePress has a premium extension called row filtering. So now I have pasted ALL of the info for each episode for eight seasons of Castle into the same table, and now instead of saying just “show Castle table”, I also say “filter to S01”. Still all one line.

Now I could get really aggressive, and paste all my shows into one database. Dozens of shows, hundreds of seasons, maybe even thousands of episodes and then filter on “Castle” and “S01”. Yet it would generate a HUGE table in the database. If it corrupts, I’m toast, I’ll lose everything, plus it would be loading the whole table each time it ran a filter. For TV episodes at least, I’ll keep it to one table per show. But once I’ve pasted one season in, the rest can go like gangbusters. A huge workflow saving, and it generates the same way every time.

And it got me thinking about how to do the book reviews.

As I mentioned earlier, I had 7 pages of book review indices generated relatively manually. Now they could be all in one page. Great! Except that all of my existing book review pages have a small table at the end of each that has links to each of the seven pages. All nicely formatted; all no longer needed. In 180 book reviews. The ones I updated 6 months ago to fix the problem with showing the pictures of the book covers. Dang it.

Editing Book Reviews

It really isn’t as bad as it sounds, maybe an hour or two of dedicated processing to open the page, go to the bottom, paste a new line that only links to the main index, and then delete the table for the rest. Easily doable. There likely is a way to do this in the block editor to prevent ever having this need again (i.e. perhaps I could edit the block next time and delete or update all of them at once), but I am not a block-editor kind of guy. I vastly prefer the simple classic editor. So that’s what I’ve done. But I went through my layout in detail asking myself if there is ANYTHING else I might want to change as I go. My ratings show as pictures of a frog reading on a lilypad, and if it is four out of five, it shows four green ones and one grey one, for instance. On all of my other reviews, TV / Movies / Music, I’m switching my ratings from an actual graphic file over to a simple icon / emoji of a smiling frog. So four green frogs and one grey circle, for 4/5. It looks simpler, shows up cleaner in tweets and FB, kind of cute. I like the branding. But for my book reviews, I like the graphic of the frog reading. So I am committing to that staying. I’ll use the frog emoji in tables, like above, but for the rest, it is graphics.

While I was playing with this, I also adjusted my movie reviews, even though for that too there are only a handful. Too hard to do the workflow, or so I thought. Now that I have an easily updatable table, it’s not that bad.

My other big tweak

A few months ago, I started the process of switching all of my photos from a separate Piwigo install on my website into a WordPress-based NextGen Gallery that embeds all the photos into the site. The integration is great, but it is a LOT of work to move 13K photos from one server area to another. I’m fixing a whole bunch of stuff on the back end as I go, including how filenames and captions, plus face tagging, are done, and I’m using Mylio as by desktop photo processor along with its built-in facial recognition. That has a small impact on my movie and TV reviews as I do include some photos for those (like the show’s title screen and a pic or two from an episode somewhere in the season). It’s working well, but I’m a bit stalled on the “big” move. Still a LONG way to go on the regular personal photos, not to mention astronomy photos later. Yikes.

Conclusion

And that’s where I am. TablePress as a major change, plus its extension for filtering + I’ve reformatted the entire approach to reviews + I’m using a new gallery plugin on the backend. But I’m really happy with the approach, for the first time in a long time. I feel like there aren’t any niggling elements on any of the review contents, or the others really, where I don’t have the approach I want. No “unresolved” issues like manually having to do multiple index pages rather than having the system generate it for me.

Yay me!

Safety in a box

The PolyBlog
January 15 2020

I have a decent physical backup plan for my existing desktop and files, although primarily I’m interested in the safety of our family photos in digital format, the best of which are also shared to my website in full original size. So I haven’t felt the need for a separate cloud-storage option for those. Other files are either backed up through email or on other backup drives.

But one downside to backup drives is where do you keep them. Some people take them to their office. Some people have “backup buddies” the same way they exchange house keys with neighbours. Or they store them with family. And still there are those few souls who follow some good online advice and keep them with their important documents and jewelry in a safe-deposit box.

There are some unfortunate souls who say, “No problem, I keep my backup drives in a fire-proof safe” and think that actually keeps them safe. First of all, “fire-proof” is not the same as “heat-proof”. While paper is safe up until 451 degrees Fahrenheit, more or less, your hard drives will melt LONG before that. Plus, many of those safes are fire-proof but not water-proof. Fires in homes attract firefighters who soak the entire house, often deluging the safe with tons of water. The end result is a perfectly safe set of contents that never caught fire but did melt and sit in water. Neither are good for backup drives. And some of the safes come with guarantees — where the guarantee is just that they’ll refund the purchase cost of your safe if you lose everything on your drive. That first word of your child? Gone. But they’ll give you a couple of hundred dollars to make up for it. Maybe.

I admit that I have been burned in the past from not having a regular enough backup schedule, and that can still happen (I don’t do daily backups for instance). But for the most part, I won’t lose much anymore. At the moment. But my files are growing with my astrophotography work and a bunch of my current photo gallery work. So a home for my disk drives is in order, but it needs to be something that I can access regularly to do swap outs with new backups or it doesn’t work.

I have two good drives for that, small, portable, and capable of one set of backups from my desktop, Andrea’s desktop, Jacob’s desktop, and my laptop. With a bit of room to spare. And then I have a big multi-terrabyte drive that can hold lots of things, plus Network-Attached-Storage that I don’t use properly as it’s kind of slow. I don’t have an off-site location to store things with friends or family that would work, nor do I want to take it to work (although I have in a pinch). No, what I need is a paid storage option.

Enter the Safety Deposit Boxes

The most obvious solution is to get a safety deposit box that I can access easily and swap the small drives out. Or put the big one in it. The question is cost vs. the cost for a cloud storage option capable of holding everything.

While having an SDB used to be a sign of “adulthood” in a sense, it is now more like “Okay, Grandpa”. People put bonds in them (I have none), jewelry (I have none), house and will documents (kept with the lawyer), and other investment certificates. If you DO keep investment stuff in it, it is theoretically tax-deductible as an expense on said investments. I also don’t have coin collections or important business contracts. It really is just about my disk drives.

Almost every SDB is big enough to hold business size envelopes. 5″ wide is a relatively standard dimension, 24″ long, and of varying heights. My small drives are less than 4″ in width so would easily fit. My larger drive is more like 5.5″ x 6.5″, and 3″ thick. Not a great option to find a cheap box to fit. On the other hand, some banks give you a free box if you have your investments or accounts with them and maintain over a certain balance. We use a virtual bank (Simplii) so that doesn’t help, and others with CIBC don’t come with any freebies.

Looking around, I see:

  • Scotiabank: $55/year for their smallest box which would hold the small drives; the larger drive would necessitate one at $125/year;
  • TD Trust: Free if you meet certain requirements, or $60 for a small, $100 a medium, but size isn’t clear, could require a large for the big drive, $150/year;
  • RBC: $60 for small, $90 for medium;
  • CIBC: $60 for small, unsure for medium;

Apparently credit unions are often a bit cheaper although I don’t see obvious choices nearby. And then there is the off-site storage option. Places like Dymon Storage also rent them. And while they are supposed to be “cheaper”, they aren’t when it comes to the sizes I need. For example, Dymon has one size for $120 a year (and right now, if you ACT NOW!, you get 3 months free). But there’s no minimum when you sign up, $10 a month, easy access, etc. You can even insure the contents, which you can’t do at the banks. Although proving your claim is always a challenge, but I digress.

So they’re in the middle on cost between a small and a medium box, right? Not really. They’re actually a really good deal because they have decent security and access like they have for the rest of their stuff, but the box itself is 12″ x 12″ x 12″, much bigger than the basic ones at the banks. It basically holds a banker’s box size set of files. That would run you well into the $200+ range at any bank. But it is WAY more than I need.

Which means I’m in the small box option of around $60 somewhere. Or finding an alternative.

Enter the cloud servers

As I said above, I don’t particularly want to go the cloud storage route as my files are too big a collection, now representing more than a terrabyte of info. Which includes:

  • Data Drive 1 — astronomy info, creative stuff I’ve done with memes and graphics for websites, etc., documents going all the way back to my university days, all carefully organized, along with some key work things I kept, my ebook collection, music collection, some learning materials, materials about photography, backups of my website content, and, last but certainly not least, all my writing. Total size: 374 GB.
  • Data Drive 2 — mostly photos and pictures, photobooks, videos, apps I’ve installed, a backup of my music folders, and a small clipart collection that I use with my graphics. Total size: 869 GB
  • Data Drive 3 — this is mostly current stuff, bookmarks, and my “current” photo repository, which much of Data Drive 2 will migrate to over time. Total size: 35 GB.

Overall, that’s 1.278 GB of data that I would rather not pay to store as the plans get expensive above 500GB-1TB. There is some redundance built in, so maybe call it 1.1 GB in active data, but still. And it is that 1 terrabyte limit that will bite me in the heinie.

If I go with Apple, for example, their 2TB plan is $12.99 a month. Or $156 a year, although cheaper with some annual discount in there, I believe, and shareable across the family. It’s certainly a viable option.

Google Drive could be an option, although I don’t really like it for these types of things, I find it clunky. It is $13.00 a month. $139.99 annually (as I said, there’s a discount).

Microsoft One Drive is great at only $6.40 per month, but only 1TB. Pooh.

Dropbox Plus is an obvious option, but at $12.99 a month for 2TB, not ideal.

Sync.com offers a $5/TB option, but you have to have a minimum of two users so a TB each or $10/month.

Box.com offers small storage plans (100GB) and $7 / user per month, minimum 3 users. Umm, how about no?

Pcloud gets good kudos but have a $350 LIFETIME charge, but who would commit to that? Monthly is $8 with annual discount applied for 2TB.

Egnyte sounds great at $8 for 5TB except that is per user, minimum 5 users. So $40 a month. Again, how about no?

Opendrive is probably the best deal for growing data, as my astronomy is going to do for bytes. Open drive is $10/mth with unlimited storage. Not the most feature-rich though. And it has some strange wording about not storing media libraries there, although that could be more about cross-listing “public” libraries more than backups.

And then it gets complicated

You see, I have Amazon Prime, so that is already paid for. And with it comes unlimited storage for full resolution photos. Like all my astro data to come. But Amazon wants $125 for 1 TB of non-photos…so all my photos would be free to backup, but not other files. They start at $25 for 100GB. Which I would over-ride in about 10 seconds.

They also have a limit of 5GB of total video storage, but beggars can’t be choosers. Particularly if, for example, I was to use Amazon Prime for all my photos, maybe my videos, and another online tool for my other documents. A hybrid online solution. Not as neat, but potentially cheaper than an SDB or any of the other options.

Conclusion

I don’t really know yet. I have a couple of other things to work out. But if I could do online storage of my photos and videos with Amazon as an “extra” for free, and then “deposit” my harddrives at some place like Royal Bank which is two blocks from my house for $5 a month, that’s not a bad solution. I don’t think I’m ready to go all in on cloud storage. At least not until I get around to sorting those extra digital files a bit more into docs and other. My biggest worry is astrophotography. It won’t take long for large files to start overwhelming any of my existing solutions.

What do you do?

Revisiting my digital photo gallery

The PolyBlog
December 12 2019

As part of my #50by50 posts, I repatriated all my videos and pictures from SmugMug, threw them into Piwigo, and (mostly) completed a good layout and design for my online photo gallery. I had tried integrating directly into WordPress, but the biggest and best (relatively speaking) gallery called Next Gen Gallery just didn’t play well with some of my other plugins, and I couldn’t get it to work right. I tried various other WP tools, but nothing was jiving for me. Piwigo worked, I found some themes I liked, I tweaked some stuff, called it a day. Then proceeded to put a LOT of time and effort into uploading 12755 photos and videos of various types and sizes.

I made it as good as I could, but it was far from “perfect”, if there is any such thing. For example, Piwigo likes to play with different size images. So it would take the original ~13K photos and make a thumbnail for each one. Plus a medium size. And a large size. Which means ~13K photos suddenly becomes ~52K files on the site. Plus the Piwigo install itself…plugins, core files, themes, etc. Call it another 3K in admin files, and I’m at 55K for the number of files. Which isn’t a problem on the one hand — my account comes with unlimited storage space. Great! Except there’s a small caveat to that unlimited storage space. It only allows 200,000 nodes which are basically file markers. 200K nodes = 200K files. I’m only at 55K, but the wrinkle?

That’s just the gallery. I also have AstroPontiac, ManagementConsultingServices, and oh, yeah, all of POLYWOGG.CA i.e. this site within the 200K too, with separate full installs of WordPress three times (that’s another story, but still). Which at one point put me close to 150K nodes. As I continue to add and upload stuff, that “margin” starts to shrink. Not a problem “yet”, but I’m looking at expanding my online presence soon, and Piwigo is taking up a lot of nodes.

Enter a new wrinkle — or two!

My hosting provider recently migrated a whole bunch of accounts to new, larger, faster servers, and my account went with it. But after it was done, for some reason, part of my WordPress install and part of my Gallery were no longer working. This is not an uncommon problem, actually. One of the downsides of running multiple installs on my server is that a couple of key files, mostly related to security, all reside in the same directory. So three copies of WordPress and one copy of Gallery all want to play in that same directory, and they don’t all know how to play nicely. When something changes for one, it can — and does — present challenges for the other installations. The three WP sites got along fine. But my Piwigo gallery wasn’t liking the new server setup.

My hoster fixed it, great. Then it broke again when something changed. So they fixed it again, great. Then it broke again. So they fixed it a third time, and it was still broke. A fourth time, still broke. A fifth time, fixed and stayed fixed. But it required a couple of tweaks that are not optimal for site operations. Not mission-critical problems, but a small design challenge, and likely to cause me problems down the road with other plugins and operations.

So, I reached back into my blog, pulled up my musings from earlier about different plugins to replace Piwigo with the idea of trying to fully integrate into my blog, and of course came across Next Gen Gallery again. Over 900,000 sites use the plugin for galleries. And yet again, I thought, “Why won’t it work with mine?”. So I gave it another go, expecting it to fail but thinking maybe this time I could devote some time and figure out what the conflict was and fix it.

I installed NGG, activated it, tried a test gallery, worked perfectly. Wait…what?

Setting up NGG for my gallery

Yep, it works now. I think mostly because I’ve switched security plugins and now it likes my configuration. Or at least doesn’t hate it. Well, that changes things. I started playing with it, a few limitations that I can live with, and I decided to go for broke and buy the pro version. Also works perfectly. Relatively anyway.

Sure, I have to tweak it for setup to match my themes and blog, as I would with any plugin. There are a bunch of gallery themes, none that work well enough to replace my overall theme, so I can ditch those. There are also layout templates, some basic, some pro, and lots of tweaks that each one can do. In the end, I really like a first page which shows thumbnail images. My favorite is called the Pro Thumbnail Grid, lets me put a legacy caption below each photo, space them out more or less grid style, and also make them fully responsive (i.e. on small screens, you get 2 images across; on my wide-screen, I get 4; on mobile, just 1). I can set a default for most of the settings, change the colours to match my blog’s theme a bit more, etc.

And then choose from a handful of different lightbox settings (i.e. the way it looks when you click on a thumbnail and it opens the pic into a full image, complete with caption, social media sharing icons, and a place to comment on the picture if you want). I had to do styling tweaks on both to get the result to look the way I wanted it to look, but one of the benefits of having the pro version is that it comes with support. So I asked questions of the developers and they told me how to style some of my unique tweaks. Which then led me to figure out some of the tweaking on my own, a sense of accomplishment that pushes my ego button pretty hard. I was pretty self-satisfied with my initial progress, particularly as it has me doing some CSS style sheet tweaks that I’ve never really done before at this level. 🙂

There are still some formatting bugs to work out such as some styling of breadcrumbs on an internally-generated virtual page. I also found a great alternative layout to use for my astronomy photos. It includes EXIF data (camera setting info), which is helpful to see with each picture. I haven’t fully styled that page, but should be only minor tweaks once I get to my astro photos.

But wait, there’s more

One of the ongoing challenges I have always had with my images is that a lot of the data is manually entered with the pics online. So all of my so-called meta data for captions, folder names, descriptions? They exist only in cyberspace in the database of the apps I’m using; the pics themselves do not include those descriptions. Which means when they went from desktop to SmugMug, they all had to be re-coded manually. When the photos went from Smugmug to Piwigo, a small percentage of the data went with them, but most had to be manually re-entered. Now that I’m going from Piwigo to WordPress, the spectre of potential recoding rears its ugly head yet again.

But as I went through photo editors last year including looking at photo management options, I tripped over a program called Mylio. It is not the best editor by far, but it has an advantage over others. It allows you to directly edit metadata, embed it in the photo so it never has to be updated again, and when uploaded to NextGen Gallery? It can read the info and display it. Including not only captions to go with each photo but any extra “tags” I put on the photos. Sure, there are other programs that do that, but can they do it in an easy to edit “group photo” page? And more importantly, not for the blog, but for self organization, can the others do decent facial recognition? No, not very well.

Yet Mylio was one I tried before, and at the time, I set it aside for later when I plan to process some photos from my mom. But if I’m going to the trouble of fixing all the metadata — and doing it right so I never have to do it again! — then I might as well have the biggest tagging aid working properly too for my own photos. Booyah!

But wait, there’s less

While having Next Gen Gallery working and using Mylio to organize the photos before uploading are great, there’s always a catch, right? Of course there is. NGG doesn’t manage videos.

Crickets. Chirping.

So? So, I have a fair number of videos of Jacob, for instance, embedded in my current gallery. Which works REALLY well, and I like it. Alas, NGG won’t handle video. And 18 months ago when I ran through a whole whack of gallery options, if it didn’t have an option for video, I killed it right away. 18 months later, I’m not as fussed about that. I can find work arounds, as long as I have a really decent photo gallery working that is fully integrated with my website. I have a couple of other plugins to automate my video management for me, but otherwise, it’s all good to go.

But wait, there’s work…lots and lots of work

Yep, it is work. Work that I’ve done before, in a sense, but I can re-use that work from before. Captions, album descriptions, consistent workflows, etc., it’s all saved on my site. So much so, that I have it nailed as a twelve-step process to get a gallery (what I used to call albums) up and running (a single month is a gallery, for example). Here is the process:

  1. SORT THE PHOTOS — This is MOSTLY already done. I have a good file structure that distinguishes between “extra” photos and what I consider “production” photos, i.e. the ones that I’m willing to share. Sometimes that might be 10 group photos where only one has everyone looking the right way. The other nine go in a sub-folder called EXTRAS, the good one goes in a root. But I do have a bit of tweaking here and there to do for the files, such as breaking really large galleries into 2 or 3 by event rather than just dumping the whole month in a single folder. Sometimes that is either a special trip during the month to, say, Toronto or Montreal; in another, I have 6 folders of day to day life doing various things, and 1 folder of a wedding with 100 photos of family. If I dump them all in one MONTHLY folder, it gets unwieldy to navigate. Not impossible, but a few times when I was working with the old photo galleries, I thought, “Hmm, maybe I should have organized that differently.” Now, since I’m “redoing” some of it anyway, I can fix it as I go. AKA the “anal retentive” step.
  2. STAGING — Before I import into Mylio, I like to make a separate copy of just the production photos and put them in a separate folder. Then my import is completely clean with no chance of huge duplicates. Nor do I end up with the videos clogging the sub-system. This also has an extra advantage I hadn’t foreseen — when I go to make photo books later, as I want to do, they will already be “reduced” down to the key ones to consider.
  3. INTO MYLIO — While this is generally a question of just importing, I also do the facial recognition at the same time. I have it scan all the photos, do the best job it can in finding faces, and then it prompts me to identify people in batches. If you think of a wedding as a good example, there will likely be a fair number of photos of the bride. Almost always in good light facing the camera. At least in theory. So Mylio is going to be able to tell her across a bunch of photos in each folder. Then it asks me, “Who’s this?” and groups every face that seems to match that same configuration i.e. all the faces of the bride look the same, and has me answer. “Jill” for example. Then it tags them all with Jane’s name. Then it shows me face group 2 — likely a slightly smaller subset of some dude’s face, in a hetero couple at least — and voila, I can tag “Jack” for all the photos that have a face that matches Jack’s in it. And it shows me the set that it thinks are all Jack so I can quickly verify before tagging them all. Oh wait, it got a slightly blurry face in there too that it thought was Jack but is really his cousin Bob. Tag that one out, tag the rest as Bob. And so on. Until it gets down to a very small set of photos that it doesn’t know who they are from the database, and not that many to group. So it gives you a photo, maybe one that has Jack and Jill already tagged, along with Aunt Martha. So now you add the tag for Martha. How well does it work? Pretty impressively actually and pretty funnily too at times. I have tagged myself at age 20 and 14, and bam, it looked at a photo of my family where I’m under age 5, and it said, “Hey, is this Paul?”. Yes, it makes predictions. On the funny side, it looked at another photo of my grandmother and thought it was my brother Mike. Another one was a photo of my dad holding a garden gnome, and the computer thought the gnome was a person — my sister Marie. I REALLY wanted to click, “Why, yes, my sister IS a garden gnome” but that seemed counter-productive for a reliable database. Instead, I can just click ignore on faces that are not actually people or even ignore faces in photos where there are 4 strangers in a street scene.
  4. MYLIO FOLDERS — The import feature is a bit tempermental, partly as I want to make sure the folder names in Mylio are consistent. So I tend to import them, and then play with them a bit. Nothing major, just some minor cleaning up in a working sub-area and then “moving” them where they should go.
  5. MYLIO KEYWORDS — I then tag a group of photos, and add some keywords. Could be simple like “Wedding, Family, Cottage” for cousins who got married at the cottage. Or could be “Trip, travel, Bangladesh” for a trip Andrea took to Bangladesh. Any photo that has identified / tagged people in gets their names added to the keywords too as these show up as “tags” in WordPress later. The benefit of that is that when everything is uploaded, I can show a “tag cloud” and click on the tags to see a virtual album of all the photos that match. Such as “Jupiter” for all my astro photos of Jupiter. Or “planets”.
  6. MYLIO CAPTIONS — While keywords handle the themes, I usually like to have captions for each of the photos that describes more specifically what is in the photo. It often is a series of captions for sub-groups of photos. Like “dancing” for a bunch of photos from the wedding. Or “skating on the canal” for four or five shots that are all about skating on the canal. However, sometimes there will be a sub-shot that I’ll be more specific about like “first dance” while the rest are just “dancing at the wedding”. I’ve often grouped photos that way in my previous galleries rather than trying to name each photo separately. It’s hard to be creative enough to say “dancing” 20 times in different ways.
  7. NEXT GEN GALLERY UPLOAD — Okay, finally, I’ve got a gallery in Mylio ready to go, and I have to upload it to WordPress. I could “cheat” and try to import the photos directly from the Piwigo site a few cyber folders away, but while it would save upload time, I’d lose all the captions and keywords. Hence why I’m doing it completely anally this time. I name the new gallery according to a very specific filenaming convention to give me an easily sortable list, add file links, and say UPLOAD. Oddly, it has a limited number of formats that it likes. Video is not one of them, and if it hits a video, it just ignores it. I would rather it said, “Hey, VIDEO HERE”. But alas, it doesn’t.
  8. CREATE A GALLERY PAGE — Now that I’ve uploaded a gallery, such as 2005-01 January, I create a page on the website to “embed” the gallery with the right layout, templates, etc. and ensuring the right file structure so the URLs have a simple and easy to navigate structure. And making sure it doesn’t conflict with my PIWIGO install. Yikes. I also add a light touch to the page with a description describing the photos in the gallery (mostly a copy/paste from earlier descriptions). Unfortunately, Mylio doesn’t do descriptions of folders, just images. So this part is still manual.
  9. ADD VIDEOS — Before finishingthe page, I upload any videos for that gallery and save them manually to the bottom of the Thumbnail page. Add a few captions, tweak the layout a bit for size of the video player, then save, publish, and the page is good to go.
  10. MODIFY GALLERY DESCRIPTION — Once I have written my little description as the intro to the page, I copy and paste it over to the actual database and have Next Gen Gallery remember it. There’s an option to display ALBUMS of sub-galleries, and I can have it both keep a description to show online in a virtual page, as well as make sure cover photos are showing for each gallery.
  11. FACEBOOK — Once the gallery is “finished” and the page is good to go, I share it on Facebook, along with the description and tag Andrea.
  12. MOVE BACKUP PICTURES — Back at the start, I moved a temporary copy into a working folder for Mylio to play with. The import process creates a copy and saves it somewhere else, so I can delete the folder. However, I do have two other folders to manage — the collection of photos that have been successfully uploaded, and the collection of photos still to be processed. Both include ALL the photos — extras, videos, maybe even some text files if there was something relevant to the “event”. All those get moved to a backup directory so that I don’t lose everything if I lose a disk drive in my computer. Plus of course all the areas (Mylio, TO BE UPLOADED, and UPLOADED) are backed up separately too.

And since I’m completely anal, I have a tracking sheet for each year. For 2005, I have 19 separate galleries to do (12 months plus 7 special events) with 12 steps above. This means a total of 228 steps to cover the year. Some are a bit time-consuming, some are pretty short. But I track them to make sure I don’t miss any steps and suddenly find myself with “some photos” processed and others in the same folders not even reviewed. Like I said, anal-retentive.

I wouldn’t say I’m completely satisfied with everything, but I do really like having it all in one install of WordPress rather than separate installs. Particularly as I often blog about events that match the photos — now I can embed the gallery and blog around the pics more easily than embedding separate photos as I had to before.

It’s a work in progress, as my blog always is. But I’m pretty satisfied with my progress so far. Ask me in a year when I get through all the galleries and get caught back up.