Big ideas about the web.


  • Some things have changed since I posted Part 1 so many months ago. I wrote:

    I’m still not sure if using Paragraphs the way I did is ideal. While it does make metadata Views-friendly, the fact that metadata labels are stored as text fields isn’t as slick as I’d like it to be. That’s also why I haven’t spent too much time refining the display of metadata.

    I since then decided that storing image metadata as paragraphs wasn’t cutting it. Paragraphs made filtering and sorting my views too complex, and although the data structure was slicker it wasn’t worth the performance loss. It also became clear that I didn’t really need most of the metadata the system was saving automatically, and I could easily narrow it down a few essentials.

    I converted the few datapoints I needed (creation date, geographic information, shot metadata) to field attached directly to the media entity, each appropriately typed, and got rid of everything else. Ultimately, each original file still retains that information, so I should always be able to retrieve additional fields if I ever need to.

    Because metadata handling was the only spot that used the Paragraphs module, I was able to remove it from the system completely. That was a relief all in itself.

  • Screenshot of the album's front page

    About three years ago I had the idea to build a replacement for Flickr. The main reason was that I wanted a place to store videos, mostly (as I was and still am in the process of digitizing old VHS home videos, which of course will never end), from which I could also easily share them. And while I was at it, I might as well just use that place to store my photos too, and consolidate all my sharing needs.

    Over a year later, after a couple of failed attempts at using different prepackaged platforms, I started building it as my first real Drupal 8 project. Of course.

    But despite all the awesomeness in Drupal 8, and above all the promise of core media, the road to getting all my requirements into place would be very long.

    Over the course of the next few posts I’ll cover all my requirements and how I solved them. Eventually, I’d like to clean up my code enough to be able to make it available on

    I wanted the system to:

    1. Allow multiple uploads
    2. Collect and save EXIF data
    3. Not make me worry about disk space
    4. Allow private albums and media
    5. Allow select guests to view any private content
    6. Allow easy importing from Flickr

    Multiple uploads

    Because I wanted to maintain perfect independence between albums and media, I couldn’t just add a Plupload widget to a node’s multi-value image field and be done with it. This for a few reasons:

    1. I wanted to take advantage of the media module (which wasn’t yet in core when I started the project, but the transition from Media Entity was seamless). This would be enough of a reason, but I had a few more.
    2. I wanted to be able to add different types of media to an album.
    3. I wanted to be free not to associate media with any album at all.

    The idea of uploading media one at a time was unthinkable—especially not after years working with Flickr’s very smooth upload experience. The only thing that would make sense would be to have an upload process that creates a single media entity for each item uploaded, and that would allow media of different types to be uploaded at the same time.

    Chance was that just a few weeks before I started the project someone had tried to tackle exactly the same issue, with the Media Upload module. I tried using that for a while, and I contributed some ideas and a few code fixes, but—long story short—it wasn’t really doing it for me, and rather than trying to hijack someone else’s plans I decided to build my own upload module from scratch. I know, not very community-oriented, but my requirement was for more than just an upload form.

    The module provides a two-step form: one step for the upload itself, and one for the details that will populate the media entity.

    The upload step uses a Plupload widget by default, because as much as I dislike depending on JavaScript it would be tough to make an HTML-only form work across various browsers. (Not that I’m going to use anything other than the latest Safari, but I’d like to be mindful of others if/when I publish the module.) This way I can also take advantage of a few more of Plupload’s perks, particularly the ability to break up large files.

    On submission, the form saves the uploads as temporary file entities and associates them with media entities. The reason I don’t just work with temporary files and entity stubs is that I wanted to display thumbnails in the next step, and as much as I tried I couldn’t find a way to make it happen unless I saved the media entities first. I will get back to that at some point and see if I can get Drupal to produce the thumbnail before saving the entity.

    To try to prevent duplicate files, I’m using the File Hash module, which does a pretty good job at figuring out which files already exist in the system, thus which media entities should not be recreated.

    The details step is a megaform that displays a few fields for each media entity created:

    • Thumbnail (just a markup element)
    • Title
    • Description
    • A choice of tags from my “Subsets” vocabulary.
    • The audience selection, which is a custom field to decide whether a media entity should be public or private. This is less granular than Flickr’s privacy settings, but I think it’ll do.
    • A “Keep” checkbox that can be unchecked if the user doesn’t want to actually save the media entity, so the entity will just be deleted. This is where it would be better if the media entities hadn’t already been saved, but presumably very few entities will not be kept after being uploaded.

    There are a few fancy things happening here:

    • The title and description fields are pre-filled using the EXIF title and caption data. If there is no title data, the file name will be used (which is kinda meh, but it works).
    • The subsets and audience settings are also pre-filled using machine tags.

    Submitting the form triggers a batch operation that takes care of saving the media entities with the new information and deleting those whose “Keep” setting was unchecked.

    Room for improvement

    Each piece of the details form is hard-coded to match the structure of the media form, and doesn’t use a custom form view mode yet. When I started building the form it was early enough in my Drupal 8 game that I hadn’t yet figured out how to pull up a custom entity form. I have since been able to do that in a different part of the system, so eventually I’ll go back and rebuild the details form in a more future-friendly way.

    Collecting and saving EXIF data

    Note: I updated this section in May 2020 to reflect a more recent solution.

    EXIF and IPTC information is fairly unpredictable, but ultimately the few datapoints I needed (creation date, geographic information, shot metadata) could each be stored in its own appropriately typed field, attached directly to the media entity. Each original file still retains all information, so I should always be able to retrieve additional fields if I ever need to.

    To retrieve EXIF values from images I had a couple of different solutions: one was the EXIF module, which is very opinionated and outputs data in a very nicely organized array; the other was File MDM, which is more complex and would have required me to massage the data it retrieves a lot more. I chose EXIF mostly because I was impatient and just wanted to get it done, but a different requirement later on forced me to also install File MDM, so eventually I’ll go back and rearchitect this part of the system to avoid having two modules that do pretty much the same thing.

    When I retrieve keywords, anything that matches my patterns for machine names (audience and subsets) gets processed accordingly, while everything else gets saved as a keyword.

    Room for improvement

    The way machine tags work is largely hard-coded to work with my specific setup. It would be fun to make it more generic and build a proper API out of it.

    Up next

    How I’m dealing with disk-space hunger and what I’m doing to provide reasonable access control.

  • A new module for Drupal 7 has just come out of our pantry, and is now live on

    Formatted Titles allows to change the type of the title field in an entity’s edit form from simple text to formatted text. When the entity is loaded, its title is then replaced with its formatted version.

    In a view, Formatted Title tries to replace all instances of an entity’s title, even when the entity is being brought in through a relationship.

    After enabling the module, go to admin/config/content/format-title, where you can enable formatted titles on a per-entity, per-bundle basis. You can also choose to restrict the HTML output of the module even further.

  • The announcement of the iPad by Apple at the end of January has fueled a discussion trend among designers and, more generally, people working with or around technology, concerning what might be called the simplicity of computing. While it’s not a new discussion, it seems to have become more and more urgent.

    (By the way, I’m well aware that what might seem urgent to me, or to other people who work in the tech field, will be of very little interest to others. I’m not blinded by the illusion that technology topics should be of primary importance to everyone.)

    I didn’t think I would write about this, because writers with wider readerships have done so already (see Fraser Speirs and, with a slightly different angle, Rob Foster), until I saw the question TUAW asked its readers yesterday, “What do you want to see in Mac OS X 10.7?” At first I dismissed it as the usual I-have-nothing-better-to-write kind of post (I was probably right on that), but then I realized I actually had my own answer.

    One idea that emerges from Speirs’s and Foster’s posts is that people spend way too much time perfecting the way their computers work, or making them work in the first place, or solving problems that, in most cases and to most users, appear completely random. The fact that most computers are operated via a graphic user interface does not make it any less complicated.

    People don’t read

    I will go out on a limb here and say that the majority of users believe computers are hostile and cannot communicate. It’s a wrong belief, of course. Computers may not be friendly, but they’re not hostile either: they exist to solve problems and to help people get things done. Also, if anything, computers suffer from an excess of communication—and I think that’s what scares people. The average user gets lost in the excess of verbal and visual stimulation, which is, even in the best of cases, never really explained or understood.

    People don’t read not because they’re lazy or stupid, but because they can’t possibly spend all their time trying to understand the machine. And yes, one would think that after more than twenty-five years of desktop metaphors, windows and menu bars some basic concepts would have started sticking—but it’s not always so. I have personally experienced frustration when talking to people who have been using computers (Macs, even) every day for the past ten or fifteen years, realizing that the word “menu” or “icon” meant squat to them.

    When I give advice or support on the phone, one of my usual questions is “What do you see?” The most frequent answer I get is “Nothing.” To me, that would mean one of two things: my interlocutor has suddenly gone blind, or the computer screen has gone black. In reality, it means that the visual overstimulation leads to a loss of understanding of a few basic elements of interface design.

    Topology (the way things are placed on the screen) is among the first to go: the user fails to understand the boundaries between things, so there are no more windows; the three-dimensional metaphor is also lost, so the user can’t see the idea of buttons being pressed or windows being on top of each other. The loss of control is so overwhelming that the user stops seeing words on the screen as meaningful entities, and sees them only as part of the machine: things the computer “puts there” but that are not really there for the user to understand or interact with.

    The designers’ failure

    The more designers understand the machine, the less they understand other people’s understanding of it. (I know that by using the word designers I might generate another misunderstanding: I don’t mean “people who draw,” I mean people who make things work the way they do.)

    Let’s face it: when the Macintosh came out in 1984, very few people used computers. And back then computers meant using a command line to interact with them. The desktop metaphor was easy—and way more essential than it is today—but it was made easier by the fact that the average computer user was already familiar with what was underneath it: the file system.

    Today, things are different. Even the most organized people I know—organized in life—sometimes cannot bother to understand how to properly use their computers’ file systems. Not even now, when a brand-new Mac comes with ready-made Documents, Movies, Images folders. Still, I don’t think that makes them lazy or stupid. It makes them people. The desktop metaphor has stopped being a metaphor, simply because people don’t see it as such. Desktop has become just a cute name someone gave to some screen on my computer, but I really don’t know why, and I don’t really care.

    But some designers do not fail

    Enter Apple. Really, it’s not because I’m completely smitten with its products that I always use Apple as a benchmark. It’s because, in most cases, whenever a new products comes out, I just feel Apple’s designers just get it. Better yet, they just get me. And with the iPhone and, now, the iPad, they do not just get me, they also get all those that are completely unlike me. (And again, designers are all those who contribute to making a product the way it is—whether it’s Steve Jobs himself and his ideas or the engineers who make it happen or the architects who make it look so damn cool.)

    Apple understands that whatever metaphor worked for the users of personal computers back in 1984 might not work anymore. Or it might work for a subset of current users—a small one, at that—but what about the others? What about those who don’t need things to be complicated, who don’t need to decide where to put a certain file, who don’t need to spend one hour trying to find the file they so carefully misplaced—what about them?

    I won’t go into the detail of why the iPad will work very well (maybe not yet perfectly) in this sense, because, like I said, the thought has already been expressed by other writers. Let me just say that the iPad aims to answer the question of “everyone else.”

    What do I want from Mac OS?

    Maybe it won’t be done in 10.7—maybe it’s something that will require a whole new version number. What I really want is a Mac OS that has two faces: one for me and one for everyone else. I mean, simply put, the ability to switch from a power-user interface to an iPad-like interface according to need.

    Sure, the Mac already has the ability to restrict a user’s power to do things, by limiting the applications he can use or the interface elements he can tweak. But it still doesn’t tone down the complexity of the interface, and sure enough it still doesn’t make the user feel more in control—quite the opposite, in fact.

    I can only imagine what such a paradigm shift would entail for all the software that’s currently available. I can imagine the resistance from certain software vendors—those that can’t do without their interminable menus with words even the savviest of techies can barely understand, or their palettes, or their multi-layered tool bars.

    I’m not advocating for a general dumbing down of user interfaces. I’m not saying computer should just be simpler. If anything, they will only get more complex. But complex doesn’t have to mean complicated. The average user will be able to better appreciate the depth and richness of a computer’s design if he can experience it without feeling overwhelmed by it.

    Mac users are always bragging about how easy to use their computers are—but are they really? To the inexperienced user they’re just as complicated (see the word I’m using here) as Ubuntu or Windows. The fact that experienced users are more willing to spend time helping friends and family set up their new Macs rather than solving their Windows problems doesn’t mean anything. Just imagine cutting down that set-up time from five hours to thirty minutes. Imagine a computer that is really understandable and usable (that is, immediately productive) within minutes after its first boot even by someone who’s never even heard the Mac chime before. Imagining it isn’t that hard now, since we know it can be done: the iPhone/iPad interface is just the appetizer.