A categorization and tagging system is not the most inherently interesting project to write about; if it does its job well, no one really thinks about it, myself included. However, it forms the backbone of the site, and DeerTrees is the realization of a vision I had for this site as far back as 2008.

Concept and History

There are many different ways to structure and organize a website's content. In the world of Django, most apps lend themselves best to organization based on content type; an image gallery, a blog, a link directory, etc. And, there was a time (before Awi 4.x) when this site worked that way too. But it's never been what I really wanted. What I wanted was a structure where I could set up global categories, and all of the site's content (gallery images, written content, links, contact links, special apps, and whatever else I add later) could go anywhere throughout the site's structure. This is partly an artifact of my strong preference toward "old-school" designs/structures, but also because I wanted to make it easy to relate different types of content together. For example, if I write something about photography, I want it to be contained in the same folder as my photo gallery. If I have screenshots, links, and write-ups for a website project, I don't want to have to put them in three different parts of the site simply because they're stored in three separate modules. Plus, separation by module/app leads to redundant site structure; what sense does it make to have similar categories in multiple different apps? That may be the easy path, but it's not the best path, for visitors or site owners.

A version of this system existed as far back as 2008, when Awi 2.x was running a PHP application. At that time, the site was quite a hodgepodge under the hood; a mixture of static content, a highly complex third-party image gallery (Gallery2), and several custom-written PHP modules. So, in an attempt to bring order to chaos, I added a seemingly simple set of new features to the Blackwolf Framework's CMS module (BwCMS): I programmed it to respond to all URLs that didn't already exist, with URL rewrite rules that expanded beyond the module's native /writing directory, and to determine its content handling based on file extension instead of URL path. So, instead of the usage of URL rewrite that was becoming popular at the time (a collection of variables presented as a /-delimited string, such as /app-filesystem-path/category/2007/01/item-slug/ or /app-filesystem-path/category/item-id/completely-unnecessary-slug/), I could fully emulate a static filesystem; all .htm, .txt, and .pdf URLs that didn't already exist as static files were handled by the CMS, regardless of directory path. I converted the image gallery to use a similar URL structure (/root-album-name/item-id.g2), and the end result was a system that, to visitors, seamlessly emulated the sort of integration I always wanted. Behind the scenes, I still had separate category definitions per module, but they could co-exist smoothly without issue.

So, for example, /photo would normally be picked up by the CMS as a category, but if I created an actual folder with that name, that folder would be loaded for that URL. Inside that folder, I added a short PHP file that acted as a "multi-root" relay for Gallery2, which defined it as a Gallery2 URL for the Photography album. Thus, when a visitor loaded the /photo directory, they got a Gallery2 photo album instead of a BwCMS content listing. All the images had URLs like /photo/22922.g2, but a link to something like /photo/gear.htm would display the CMS content as if they were pulling from the same app/module.

For such an early version of the site, with a difficult-to-modify third-party application mixed in (which was designed to be an entire separate site by itself), this was a pretty revolutionary way to integrate the various sections into a harmonious presentation. Especially since it also operated smoothly with static files (there were a few .htm files outside the CMS), and even a Wordpress installation. Since I transferred the content from Wordpress into BwCMS a few years later, this system made the URL changeover much simpler; I had set Wordpress to use the same URL structure as BwCMS (/journal/slug.htm), but only within its installation folder, and BwCMS didn't need the folder portion of the URL to load a content page. So, as soon as an entry ceased to exist in Wordpress, it seamlessly became a BwCMS page, as long as I kept the slug the same.

However, in future versions of the site, I wanted to do better. I didn't want to just emulate harmonious integration into a single site-wide directory structure, I wanted the real thing.

DeerTrees 1.x and 2.x

DeerTrees 1.0 was the first attempt at implementing my vision for Awi 4.0 and 4.1 (the site's first Django builds) back in 2012. I went for a relatively simple implementation (centralized categories that determine their contents via reverse foreign keys), with a recursively-nested tree structure as a new feature that BwCMS lacked, and while it got the job done, it quickly became unwieldy as I added more content types. Since I had no way of knowing exactly what content types would be present in any given directory, every new content type required major modifications to the views, and separate queries per content type.

I briefly experimented with using Django's Content Types module and GenericForeignKeys in DeerTrees 1.1 (part of the short-lived Awi 5.0), which caused far more problems than it solved, so for Awi 5.1 (the current build of this site), the full 2.0 rewrite of DeerTrees switched to a completely new design concept. Instead of every separate content type having a direct ForeignKey relationship to a category, I created a model called "Leaf" with basic fields needed for category-level content filtering (category foreign key, timestamps, security options). Every other model that needed to appear in a category could extend this Leaf model, which had a side benefit of reducing redundancy for those models. Then, in the views, I could just do one query (all leaves with current category as a foreign key) to get all content for the current category, regardless of type. This also made the addition of tags much simpler, since they could share a lot of their code with the categories. And, I added a built-in leaf type, Special Features, that could easily bridge the gap between the site's structure and apps that weren't integrated quite as readily; a Special Feature was simply a URL definition with a title and summary, but with all the same security/filtering, category, tag, and timestamp fields as any other leaf. So, if I had a non-Leaf app like DeerFood (my restaurant-style menu for my kitchen) that I wanted to put within the /personal/cooking/ category, I could simply create a Special Feature with menu as its slug within that category, and set the URL configuration for the app to use /personal/cooking/menu/ as its base path. DeerTrees would handle the rest, providing a link to this other app, along with breadcrumbs and other unifying features.

For the views for DeerTrees 2.0, I created a list of known content types, gave them a priority ranking for the "main" and "sidebar" display regions, and added a per-leaf getattr() check to determine which content types were present in which categories/tags. It would run through a loop to assign content types to different regions on the page, usually with the pattern "main, sidebar, sidebar, main, (sidebar then main to infinity as needed)", based on which content types were the highest priority for each region, and whether a particular content type could only appear in one particular region. On top of that, each folder could set its own Content Priority to "images", "writing", or "folder description" to override the priorities in the settings file; whichever content type was set as the Content Priority would be the first choice for the first "main" region. This system was complex, but it worked quite well in most cases, and it was the first version of DeerTrees to make it to a production Awi build.

DeerTrees 3.x

Unfortunately, the complexity of the views in DeerTrees 2.0 made it difficult to work with. The assignment of content types to displayable regions was often difficult to predict, and even more difficult to control in a granular way. It had a built-in override to display featured items from all descendant categories if the current category had no child leaves of its own, but there was no way to force it to do this on a per-content-type basis. Adding new content types had unpredictable side effects site-wide; when Sunset was deployed, it took a lot of tweaking to re-work the priority rankings, and even then, several categories ended up with layouts that just didn't make sense (like /photo putting written content in the main display region, with no actual photos visible). Plus, there was no way to add non-leaf blocks (like the special contact links that aren't leaves, and show up on all descendant categories) to the display regions of a category/tag without hard-coding them into the view. And, of course, it was slow. DeerTrees 2.1 and 2.2 introduced several improvements to deal with some of these issues, but it just wasn't enough. DeerTrees needed a fundamental rebuild, which I started working on in January 2017.

The models mostly stayed the same for DeerTrees 3.0, but with the addition of a "type" field on the leaf model, and a line in its save() method to store the name of the current class to the type field. This eliminated the need to loop over each leaf with a getattr() check per content type, yielding a significant efficiency improvement. I also replaced the Content Priority field for categories and tags with a View Type field, which gets its options from the new Block Map, forming the core of the 3.x improvements.

The biggest changes in 3.0 were reflected in its settings. The list of content types split into two separate dictionaries; one for leaf objects, and one for "special" blocks, allowing me to add custom callable objects to category and tag views. This replaced the hard-coded queries for templates and contact links with a standardized interface that could be used with greater flexibility; in addition to those two blocks, I also added widgets to DeerBooks and Sunset to display featured and recent items, and an Upcoming Events widget for DeerAttend. As long as the callable object accepts two optional parameters (parent and parent type), and returns something that the associated child template can parse (or False if there's nothing to display), any callable can be added to a category or tag view this way.

The other major change in the settings is the aforementioned block map. Instead of a list of blocks with priority rankings for each region, there's now a list of map definitions. Each map is a dictionary with regions as keys, and a list of blocks (content types) in priority order for each region. Each block will only be used once, so it can appear in multiple regions, and only the first one that isn't full will contain it. For example, the default map looks like this:

'default' : {
	'main_left' : ['image', 'page', 'category', 'link', ],
	'main_right' : ['page', 'category', 'link', 'image', ],
	'sidebar' : ['contact_link', 'special_feature', 'category', 'page', 'link', ],
},

This is much easier to predict; if a category or tag using this view contains images, pages, subcategories, and links, the images will go in main_left, the pages will go in main_right, and categories and links will go in sidebar. If there are no pages, categories will get promoted from sidebar to main_right. Contact links and special features will only appear in the sidebar, and always at the top.

Combined with some improvements to standardize the way data is passed to the template, the result of this upgrade is a powerful layout system that can display all sorts of content contained within the same category or tag, in a way that's far easier to manage than anything that's come before it. The assignment of content to regions within a page is much easier to read and predict. The new system can accommodate new regions quickly and easily (such as the new main_left and main_right that allow for a vertically-split main content region). Content can be displayed in new and interesting ways, like the new subcategory thumbnails option. And, most importantly, the display of content can be fine-tuned per category and tag simply by defining a new block map. It's not often that I get to describe something as a perfect realization of my vision, but after 8 years of iterating closer and closer to it, DeerTrees is now exactly that: a perfect implementation of the way I always wanted this site to look and function.