Over the course of my career I am repeatedly confronted with a question. The question is how to measure content quality. Whether it’s my content that I’m looking at to improve or content of someone else I’m providing feedback on, the question of content quality keeps coming up over and over again.
Recently, I was speaking with a content creator. He’s an accomplished creator of training solutions. He told me that he focuses on his content and largely ignores the production aspects of creating the content. Visuals are someone else’s problem – someone who is more gifted in these areas than him. (I’ll save the argument about giftedness for books like Mindset.)
As I ran into the problem this last time, I realized that I break content quality into three dimensions. These dimensions are distinct. Being good or bad at one has little or no impact on the others. Together, they’re how you create truly amazing content. The dimensions are:
Quality – How is the content produced? For a book, this might be typesetting and writing. For a video, it’s video and audio quality.
Structure – How is the content structured to accomplish the goal. If the goal is persuasion, how many people actually buy the product? If the goal is education, how many people learned?
- Delivery – How well was the content delivered? What’s the style, appearance, and approach of the presenter?
Let’s look in more detail about how I think we can distinguish content quality.
What defines production quality varies widely by the medium being used to deliver the content. Some mediums offer relatively little in terms of ability to improve or remove quality of the content through the production process. However, there always seems to be at least some aspect of perceived content quality driven by the production aspects.
For instance, in traditional book publishing, there’s a typesetting or layout department that manages the process of taking the stream of words and putting them onto pages. Even after the advent of programs which are, for the most part, capable of laying out content automatically, traditional book publishers have kept people who are skilled in laying out the copy of the book.
There are simple things that they must consider including ensuring that the page numbers are correct and are on every page. Nearly every piece of software for layout now handles widows and orphans. Collectively, these are one sentence either above or below the page break. Automatic kerning brings together the letters ‘V’ and ‘A’ to reduce the horizontal space between them.
These are the technical details of book production. They’re the details that will help the reader believe that the book is a quality book. It’s entirely independent of what the author is saying, but it makes a difference to the reader in terms of how professional and complete the text feels.
In video production, the technical aspects are much more challenging. On the video side, there’s lighting to consider, as well as backgrounds, depth of field, and more. On the audio side, there is signal-to-noise ratio, unwanted reverb, tonal balance, and a plethora of other considerations. (See My Video Studio 2.1 for just some of the things that I’ve done to improve the production quality of our video work. I say “some”ecause there are more upgrades that I’ve not had time to document yet.)
Ultimately the production quality of content can detract from the core content itself. Many people are distracted by typos, misspelling, and bad grammar – and it impacts the ability for the content to do its job. If there’s a hollow sound to a recording, it will take away from the perceived authority of the person speaking – and thus limit the impact of their content.
The core of content quality is the content itself and its structure. Does it accomplish its purpose in you? If the content is designed to propel you into a philanthropic cause, do you rise up from your chair and volunteer? If the content is designed to educate, who understands at the end of the day? Does the content reach the target audience? Does it positively impact a broader audience?
There are two key differentiators inside of the content structure. The first is the content’s type. Is it a piece of persuasive content designed to cause action, or is it educational content designed to improve knowledge and thinking? The second differentiator is the audience. The type of audience and familiarity requires different approaches, and by tailoring to the audience (or not), the content’s subjective quality can be dramatically improved or removed.
Rhetoric is persuasive talk. But that’s not the only thing that talk can do, as evidenced by the number of live, instructor-led educational events in the country. When evaluating the content, it’s important to ask what was the content intended to do – and then did it hit its intended target. The way that you evaluate content depends upon whether it’s designed to persuade or invade the mind.
Creating persuasive content isn’t really my “thing.” I can and do write copy that’s designed to market and sell my services. However, I’m far from considering myself at an expert at this. I don’t create commercials. I don’t create printed ads. However, I do know that the true measure of this kind of content is the conversion rate. Whether it’s a billboard, a commercial, or marketing copy, the true measure is how many people take the action that you want.
Books like Demand speak of segmenting your market and removing the barriers to their purchasing. Guerrilla Marketing pushes you out of your comfort zone to try things to create persuasive content that are unconventional and can be done with minimal investment. Sally Hogshead in her book Fascinate seeks to help you find the unique trigger in your creation that will resonate with others. Brand is a Four Letter Word seeks to focus your entire brand message into a single idea that will create feeling – and therefore action – in your messages. In Slide:ology, Nancy Duarte seeks to help you find a way of expressing yourself in slides and speaking that drives folks to action.
Despite the work on how to create persuasive content, the best advice I’ve ever gotten – from a long time personal friend who teaches marketing at a collegiate level – is, “Just try stuff and see what works.” Needless to say, it’s hard to learn the skills that lead you to great content creation when not even the experts seem to really know what it takes to be effective.
It’s not that we don’t understand the ultimate measure of persuasive content, it’s just that we don’t have consensus on what the factors are that lead to success.
I’ve spent a considerably larger portion of my professional career working on the creation of educational content. From book writing to course development to productivity aids, I’ve looked at education from nearly every conceivable angle. Despite my effort, research, experience, and desire, I still don’t always get the educational content that I create right.
One would think that, with 25 books on which I have author credit and another 100 that I’ve got some sort of an editor credit on, I’d be able to express my thoughts in a meaningful way, but you’d be wrong. My blog posts are edited by my office manager and it’s pretty routine to have her send me a comment along the lines of “I don’t understand what you’re trying to say.” In my statistical sample of one person, I’ve failed to get my point across and to educate. I’m grateful that I have someone who helps me be more successful in my written word.
The guidelines for the development of educational content have been well studied. Benjamin Bloom and his colleagues put together a taxonomy for the creation of educational content back in 1956. Since then, many have researched the educational process. There have been many books about learning and high performance. Efficiency in Learning is a survey of research about educational effectiveness. The Art of Learning
is Josh Waitzkin’s story about how he became among the best in the world at chess and martial arts, and what he believes it takes to learn to be the best. The Art of Explanation focuses on how to explain things in a way that people can remember. Source of Power explains tacit knowledge and the mental models that practitioners build – and that, as educators, we seek to build more quickly and completely.
The ultimate measure of educational content is the effectiveness at creating learning in the student’s mind. We’ve discovered that assessing a student’s learning is a tricky business that can often interfere with retention. As a result, it’s often difficult to assess the effectiveness of learning. (You can find out more in my whitepaper, Measuring Learning Effectiveness.)
Despite the fact that education is a large market and it’s been studied for a long period of time, there’s still quite a bit of debate about what good content is. In part, this is due to the fact that education and learning are largely situational. What works for one medium and with one set of students doesn’t work at another time or for another set of students. Malcolm Knowles and his colleagues describe The Adult Learner and how the adult learns differently than a child. The inner state of the learner – what they bring to the table – is more important in some cases than the content itself.
Thus you can’t evaluate the quality of content without knowing the audience for which it’s intended. On the persuasive content side, much is made about marketers and their ability to identify their target audience through workshops, discussions, and trial marketing. Knowing the audience means getting a real “sense” for them. This can take the form of a persona – a prototypical person with whom you can relate – or it can be basic observations about how they interact.
As I speak to audiences, I can tell you the feel in a room in Washington, DC is different than a room in rural Illinois. The folks in Illinois are – for the most part – more open, friendly, inquisitive, and willing to engage. That doesn’t make them more or less smart than a Washington, DC audience. It just makes them different.
I do a lot of conference speaking and each conference has its own feel too. Some conferences are flexible, open, and casual while others are more formal and “starched.” Knowing the conference allows me to know the audience and tailor the content to the audience so that it will resonate.
Knowing your audience can be knowing their age, their interests, their challenges, or a million other potential facts; but ultimately, knowing an audience allows the content creator to identify content that will resonate with them and will improve the end goal of persuasion or education.
Putting well-crafted content into an excellent production facility isn’t enough. The best joke told in the best comedy club in America will fall flat if it’s not delivered correctly. Delivery is the last dimension. That is how the content is actually delivered. Some ideas never see the light of day because their authors are unknown. Some ideas sit on shelves percolating because their creator doesn’t feel like they’re finished – or because the publication of the idea isn’t notable enough.
Delivery can mean distribution. It can mean the ability for the idea to get to the people who would be interested in it. A conversion rate for persuasive content isn’t anything if no one sees it. Educational content isn’t going to educate if no one sees it. However, delivery is more about how the content is delivered.
In the written and spoken word, this means the vocabulary and structure that is used. Is it inside of the vocabulary of those to whom it is targeted? Is the language of active voice? Does the language tantalize the emotions as well as compel our reason? Is the structure different enough to not be dull and boring, and consistent enough to ease understanding?
The spoken word also needs the force of the speaker’s voice. It relies upon the intonation, cadence, and pauses. When delivered live, the words need to align with the speaker’s body language. Body language can amplify the emotional context of the words — at the risk of detracting if there’s a mismatch.
In every dimension, finding the distinction between style and a structural problem is a key challenge. In the spoken word, it’s generally accepted that the filler “um” is to be avoided in public speaking. It indicates that the speaker is trying to pause the listener while they collect their thoughts. However, the occasional “um” isn’t a challenge, particularly when used to signal a transition in topics or a change in thinking. Reverb can provide warmth to a voice. Too much reverb makes a voice muddy and unintelligible. Quick-paced content can be exciting and engaging for a student; or it can be overwhelming for non-native speakers, whose attention is split between listening to the language and understanding the concepts.
Are the clear lines between specific educational goals more important, or is it more important that the education follow a storyline that arcs through all of the content? As you watch various TED Talks, you’ll see numerous different styles of speakers and presenters. If they’re on the TED stage, they’re invariably a good or great speaker delivering a compelling topic. (If they weren’t, they wouldn’t have been selected – but we should make allowances for a bad day.)
All too often in the content creation business, the criteria we use is subjective and based on the relatively limited knowledge of the next person. Rarely does the process encourage learning about the process. Authors are told to do things different. They must either do it – or fight until they get their way with the content. In either case, neither party really expands their understanding to the other’s point of view or sensibilities.
There are places where content quality can’t be measured. There’s no quantitative measurement for the fit of an analogy or the speed at which a particular topic should be delivered. However, there are numerous quantitative measurements that can be put in place to monitor some of the aspects of quality discussed above. Written word can be subjected to reading scores. Spoken words can have the filler words counted. Audio can have a signal-to-noise ratio checked.
Where there are quantitative mechanisms for measuring content quality, they should be used. Where there aren’t quantitative measures for content quality, dialogue should be considered as a way to elevate everyone’s understanding of the variables and conditions, so that everyone learns of different experiences and perspectives and are able to bring that to the next situation.