kdpainesocialmediawhitepaper

8/3/2019 KDPaineSocialMediaWhitepaper

http://slidepdf.com/reader/full/kdpainesocialmediawhitepaper 1/23

The Social Media Measurement ManifestoYes we CAN, and already ARE measuring Social Media

By Katie Delahaye Paine"If we can put a man in orbit, why can't we determine the effectiveness of our communications? The reason issimple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice.Unpredictable, cantankerous, capricious, motivated by innumerable conflicting interests, and conflicting desires."Ralph D. Paine, Publisher, Fortune Magazine. 1960 Speech to St. Louis Ad Club

In his paper "Power, Pleasure, Patterns: A Conceptual History of the Mass Media in the United States," JoshuaMeyrowitz of the University of New Hampshire suggests that every new medium, from the printing press to theInternet, creates a power struggle over access to wealth, resources, and power. Guttenberg's printing presswrenched the power of the written word out of the hands of the Church. Several centuries later the broadsheetnewspaper helped foment the American Revolution. In the 1830s the Penny Press brought the news to the massesand began advertising-based publishing. In 2008 and 2009 Twitter helped spread the word -- if not actuallyfomenting the revolutions in Moldova and Iran. Every day blogs are wrenching power away from corporationsand putting it into the hands of individuals. Each century has seen its media revolution and social media is clearlythe media revolution of this new century.Thanks to advances in technology that have made it incredibly easy and virtually free to publish, social media isturning the communications world upside down. To paraphrase Ken Kesey, in today’s media environment theinmates are now in charge of the asylum.Thanks to advances in technology that have made it incredibly easy and virtually free to create content,consumers, the media, and everyone in between are creating content at unprecedented rates. A new blog is created

every minute, and at one point during the recent Iran election controversy 200,000 tweets an hour were beingposted to social micro-blogging site Twitter.People -- your customers, your employees, your stakeholders - are now the media, the editors, and the reviewers.They are in control and they’re going to let you know what they think by voicing their opinions and changingtheir behavior. So, despite the best efforts of PR and marketing types, consumers will continue to seize powerfrom those in control, be they dictators, CEOs, PR professionals or marketers.

A new mindset is needed

From a measurement perspective, this social media revolution requires an entirely new way of thinking about

what we do and how we define success. We need to look at these changes as we would an earthquake. Thetectonic plates of marketing and communications are shifting. As a result, some institutions will collapse andothers will rise from the ruins. The degree to which we can get our minds around these earthquakes will be thedegree to which PR and Marketing professionals will survive.There are three dominant areas in which mental earthquakes have occurred:1. Timeliness has been redefined2. The role of PR and Communications has been redefined



3. Most definitions of success are irrelevant

Necessity #1: The need to redefine “Now”

The most obvious change in how we approach communications today has to do with the definition of timeliness. It used to be that if you got back to an irate customer in a day or two, you would be consideredresponsive. In social media time, that is simply unacceptable.To quote author Shel Israel, who was commenting on a minor squabble between prominent blogger RobertScoble and social network Facebook, “It’s been three hours and there’s yet to be a response fromFacebook.”Domino's communications executives made the mistake of assuming that Friday was the end of their workweek. As a result, a revolting video about conditions in one of their restaurants went viral. By the time theyresponded 48 hours later, millions had already watched the video and sales had already started to plummet.Frank Eliason changed forever the definition of customer service when he launched his Twitter account@comcast cares. Through Twitter, Comcast can respond instantly to customer complaints or questions. So,

when I found myself without cable TV one Friday night in NH, I tweeted @comcastcares to see what theproblem was. Within 5 minutes they’d checked my service, informed me that it was not an outage or othersystem problem, and suggested a fix. Exactly eight minutes after my initial tweet, I was watching my favoriteprogram.There’s no shortage of examples of how quickly reputations can be made and/or destroyed in today’senvironment. The lesson to be learned is in the redefinition of "timely." I suggest that timely today meansyou dealt with a problem before it goes viral. Ideally, before more than a few friends of a social media userhave seen it.Now, back in the olden days, we at KDPaine & Partners thought it was pretty cool that we could actuallydeliver reports to our clients within a week or two of the end of the month. After all, in my old company the

process took anywhere from 6 weeks to two months. Today a two-week turnaround time looks downrightquaint.The new social media environment means that, to be effective, brand monitoring must be at least a dailyprocess, if not hourly. More importantly, if there is a crisis, you must be prepared with a very rapid response.You need to have systems already in place to identify the new brand influencers, you must be aware of theissues and topics they will respond to, and you must have an internal action plan from which to operate.

Necessity #2: Redefining PR, Advertising, Marketing and Corp. Comm

New decision making processes are changing the role of marketing and communications, which brings us tothe next big required shift in our thought process: Search and reputation-based decision making. In today’s

environment, most decisions are influenced in some way by Google, Yahoo, Bing, or whatever your favoritesearch engine is. Trying to decide what flat screen TV to buy, where to go for vacation, what movie to see, orwhere to make reservations for dinner, chances are you’ll either ask your friends on Facebook or Twitter foradvice, or search on Google for reviews.When you Google a brand or a product, you may pay attention to some paid AdWord results, but chances areyou’ll mostly click on and read reviews by someone whose opinion you respect -a friend, a colleague, orperhaps a brand name media site. And chances are you’ll make a decision, based not on the number of timesyou’ve seen the brand, but based almost entirely on the reputation of the source of the opinion. "Is it



someone like me with similar needs? Is it a media person or a Twitterer or a Facebook friend that I trust?"As an example, our business recently conducted a review of email marketing service providers, consideringEmma, iContact, and Constant Contact. All our research was conducted online and our ultimate decision wasbased on feedback we heard from customers on blogs and Twitter. We did not speak to a sales person untilafter we'd made up our minds.

This process is being repeated millions of times every day. So if your brand has been talked about anywherein social media, chances are good that your potential customers are reading about it.What this means for PR and marketing is that communicating to smaller numbers of the right people is farmore valuable than reaching millions of faceless eyeballs via advertising. Consequently, the value of advertising is declining, and the value of friendships and contacts and engagement is on the rise.Unfortunately, most PR people still envision social media as yet another new medium to address, a new wayto scream more loudly at their stakeholders. However, in today’s environment, the notion that PRÕs role is toreach millions of eyeballs for free (or at least more cheaply than advertising) or to generate quantities of column inches in newspapers is about as useful as a buggy whip.The rise of social media makes the role of public relations more important than ever. PR must do what itsname suggests: Nourish and cultivate relationships with publics. In order to encourage those relationships,

PR’s role needs to shift to one of listening to the stakeholders, acting as their champion, advising internaldecision-makers, and building engagement with the brand.

Necessity #3: Changing how we quantify success

Which brings us to the final mental earthquake we need to get past: How we define success? For years,marketers have equated success with the number of Impressions or Eyeballs they’ve reached. (This wasbased on evidence that if you reached enough eyeballs you could convince people to buy your product. Thispractice worked well in consumer packaged goods, and the assumption was picked up by business to

business marketers.) So you either bought those Eyeballs via paid advertising, or earned the eyeballs bygetting the media to cover your story or brand. And you often judged success by the number of impressionsyou collected.But in the social media environment, the sheer volume of impressions is no longer what really counts. Socialmedia encourages the development of relationships between people and products and/or organizations. Andmeasuring these relationships, often by assessing engagement, is the key to quantifying success in socialmedia.

The New Rules for PR and Social Media

The nature of social media means that PR people need to completely rethink their approach: From pitching tolistening, and from measuring eyeballs to measuring engagement.

New Rule #1: It’s about reaching the right eyeballs, not all the eyeballs.

For the past few decades, PR was measured by counting the number of eyeballs it earned in the media. Thereare two problems with this approach.



Problem #1: There is no market for your message.As David Weinberger told the 2007 New Communications Forum gathering in Las Vegas, “There is nomarket for your message.” People now have access to so much content, and have so many ways to gathernews and information, that the likelihood of your corporate message penetrating the clutter is virtually nil. Soit really doesn’t matter how many people you've reached with your message. What matters is what people doif and when they hear that message. If you listen hard enough to learn what the social community is looking

for and concerned about, you might be able to persuade them to hear and respond to at least part of yourmessage.

Problem #2: The numbers aren’t there.We have to accept the notion that it’s become virtually impossible to count eyeballs. More and moreadvertisers and media types are realizing that hits really does stand for "How Idiots Track Success." Evenunique page views are highly suspect, given the enormous variation in such statistics. Influential blogs do notroutinely provide circulation figures or any other data for that matter, on how many people actually visit theirsites. Again: It’s not about how many people visit a particular site, but what they do with that information. If they just read the headline and move on, have you really made an impression? What you want them to do isread it and then pass it on to their friends and colleagues, or comment on it or sign up for something. Andthose are the things you want to measure.

New Rule #2 : It’s worse to not be talked about at all.

If your customers are making decisions based on what their colleagues, Tweeps, and Facebook friends aretalking about, it may be more harmful to NOT be talked about than to be discussed in negative terms. If someone puts something negative out about you in social media, chances are good that someone else willleap to your defense. It’s probably more damaging to your business if the market is talking about yourproduct category and no one is engaged enough with your brand to care about it.Put another way, if you’re not part of the conversation, your enemies will be. If you haven’t participated in

the conversation, fewer people will be engaged with your brand. And, if you’re not listening to your market,you’ll miss not just the damaging conversations, but the conversations that can give you insight into theminds of the marketplace.So, for example, I’m trying to decide if I should upgrade my laptop. If I Google Lightweight laptop I get waytoo many responses. If I go to TechCrunch and Engadget, there are still too many conflicting responses tomake a decision. So, I questioned my Tweeps and Facebook friends. 75% of the responses said, Get a Mac.15% said, Stick with your Dell. A few people said, "I love my Vaio" There was one vote each for HP,Lenovo, and Asus. And NONE at all for Acer, Toshiba, Panasonic. So why would I even consider them?Another major reason to pay attention to what people are saying about you in social media is that most journalists today rely on blogs for story ideas, to check facts, track down rumors, and to investigate scandals.If you’re not being discussed, you are probably not news.

New Rule #3: You’re not in control-- and probably never have been.

The normal maxim for measurement is, “If you can’t measure it, you can’t manage it.” The nature of socialmedia renders management both impossible and undesirable. You simply can’t manage what billions of independent, opinionated people are going to say. And woe to those who try; social media can resemble a



cornered porcupine, very prickly, when it senses someone trying to control it.That’s not to say that conversations and relationships on the Internet can’t be influenced, just that it takes anew approach. The old command and control, top-down message delivery is no longer an option. JamesGrunig's Excellence Model of two-way synchronous communications (Grunig and Grunig, 1992) is the ruleof the day. Consumers can now choose to accept or reject your messages, depending on whether they find

them useful, interesting, or relevant. And, they’ll be more than happy to tell you what they like and don’tlike.

How to Measure It: Six simple steps to a perfect online measurementprogram

The biggest challenge to a researcher on the Internet is the sheer enormity of the task. The good news is thattechnology can help you find your way, and there are dozens of organizations out there that will be happy to assistyou in gathering your data.The bad news is that the data you gather will probably have major gaps in it and may be of questionable validity.

Even the most comprehensive search firms can only gather a fraction of what you really want. Automatedsentiment analysis is only about 60-70% accurate. Organizations like comScore, Neilsen, Compete, and Alexa allhave vastly different data. Also be aware that media sites that require subscribers to log on will not be included inmost Web searches or in the Impression counts."That having been said, here are six simple steps to a perfect online measurement program:

Step 1: Define the goal

The first and most important question to answer is: What’s the objective? Why are you starting that blog orFacebook page or on-line community? What problem are you trying to solve: What do you hope to

accomplish? And more importantly, what does Senior Management expect social media to do for theorganization? If they don’t know why you’re doing it, you can’t implement a measurement program. So youneed to define, and management needs to buy into, the expectations and definitions of success.

So answer before you do anything, answer the question: How will you define success: Almost all answerswill fall into three broad categories:

Category 1: Sales or Marketing gainsIf your answer is “Gain market share” or “increase donations” or other such quantifiable revenue goals,then you will need a set of measurement rules and tools that emphasize Web analytics. You will need touse unique URLs and some form of statistical analysis of traffic and response data.

Category 2: Increase EngagementIf your answer is “increase brand engagement,” then you need to establish how you are definingengagement. For some that means click-throughs and downloads, for others it means comments andretweets. For still others, it’s getting the visitor to leave an email, register for more information, or signup for content. Procter & Gamble has challenged the media in which it advertises by indicating it will, infuture, only pay for engaged visitors. So you can look forward to more specific definitions of engagementin the future.



Category 3: Improve relationships or reputation

If your answer is “improve/protect our reputation” or "increase brand loyalty," then your measurementtools need to include surveys, content analysis and relationship metrics.And if you can’t identify a clear tie between your organization's goals and what you are doing in social

media, then stop reading this article. Go learn about measuring more relevant media. As in anycommunications activity, if it doesn't support a specific corporate goal, why are you doing it?

Step 2: Identify your publics and determine how your social media efforts affectthem.

The first thing I say to clients who ask me if they should worry about Twitter or whatever social medianetwork is top of mind that day is, “Don’t ask me, ask your customers.”The most important factor in measuring success is the response of your stakeholders and customers. So first

identify those specific audiences that you are trying to reach with your program. If there are too many tomeasure all of them, then make a list and prioritize them based on where your organization will see the mostbenefit.The next step is to make sure you can draw a connection between your efforts and the stakeholders you aremeasuring. How does your social media program impact them? How does their behavior affect yourorganization? If, for instance, there is a ban on communications folks talking to customers, it’s going to behard to make that connection. On the other hand if you are on Twitter and talking to customers on a regularbasis, then there will definitely be a link between your efforts and customer attitude and/or behavior.

Step 3: Determine the specific KPIs by which you will define success.You become what you measure, so pick your key performance indicators (KPIs) carefully. KPIs, also knownas scorecards, dashboards, cockpits, and a host of other buzzwords, are specific metrics by which you will judge your success each week or quarter or month. Because you become what you measure, it is critical tocarefully choose the metrics by which you will track your success. Presumably the KPIs you select are themost important metrics for your organization.

Again, which metrics you chose should be driven by what goals you established in Step 1:

If your measure of success is sales:

Sales and donations and memberships are the easiest form of social media measurement. Your metricsshould be something like:

% increase in conversion rates% increase in click-throughs to a specific URL% increase in conversions% increase in on-line donations% increase in membership signups



If your measure of success is brand engagement:

Like most other buzzwords, "engagement" has come a long way from its original meaning of "an agreementto marry." Essentially, it started with the notion that a website or a blog was "engaging" enough to get areader to begin to develop a relationship with the brand. People began to speak of measuring engagement --

not just how "sticky" the site was, but the extent to which it enhances the relationship between the user andthe brand.Communications professionals and marketers now want to measure a site's ability to create an experiencethat earns a visitor's loyalty and, with luck, its business. As a result, "engagement" now means everythingfrom the number of times that a visitor returns to the site, to the number of comments on your corporate blog,or the number of retweets of your Twitter stream.Another way to think of engagement is as the fourth step in a five-step process that the individual user goesthrough:

1. Finding, (usually by search)

2. Lurking

3. Participation

4. Engagement

5. Relationship or outcome.

Popular blogger Robert Scoble has suggested that engagement is a valid measure of user interaction and of the authority of Internet-based social media channels. That is, engagement is a way to determine whether youare really having a dialog, or you are just yelling ever more loudly. His premise is that by measuring activityon a blog or social media website as a sign of engagement, you can predict users' behavior. In other words, if they come back to a corporate blog over and over again they'll eventually buy.So metrics for engagement typically look at the ratio of posts to comments. For example, if it's a YouTube

video, do they just watch, or do they rate it or comment on it? The assumption being that they are then morelikely to pass it on to their friends and maybe even take some other action as a result.Brian Haven of Forrester Research and Eric Peterson of Web Analytics Demystified picked up on Scoble'spremise and proposed measuring engagement based on a variety of tangible and intangible factors includinglinks, comments, and the frequency, sentiment, and tonality of comments. They define engagement as thelevel of involvement, interaction, intimacy, and influence an individual has with a brand over time.Specifically Peterson defines engagement as:

• % increase in unique visitors

• % increase in visitors that return more than 5 times

• Increase in number of different pages viewed

• Increase in number of catalog requests or downloads

• Number of sessions that include more than five page views

• Number of sessions that are greater than five minutes in duration

• Number of visitors that arrive at your site from a Google search, or a direct link from your website

• Number of visitors that become a subscriber or provide an email address



• Haven defines engagement as “Intimacy” The sentiment or affinity that a person exhibits in thethings they say or the actions. So your metrics might for intimacy include

• % recommending the brand

• % of posts or comments that would be considered a shout out or endorsement

The good news is that most of these metrics can be gleaned from Google Analytics or any other Webanalytics program. The bad news is that most of this data is only available for your own site, and not for yourcompetition. So there is no way to compare your engagement with that of your competition.

If your measure of success is "improved relationships"

In many ways engagement is just another way to express the improvement in relationships and reputationthat is the goal of any good communications program. Improved communications should lead to greater trust,improved satisfaction, and more commitment to the relationship. All of these can be measured through asurvey of your audience providing in metrics such as:

• % more likely to express satisfaction

• Improvement in willingness to recommend (Net Promoter Score)

• % increase in trust, commitment or satisfaction score

• % willing to buy or do business again (renewal rate)

If you incorporate some sort of relationship metrics into your dashboard, then it makes it much easier todiagnose the underlying cause of problems. Even though Web metrics can track behavior with increasingaccuracy, all the Web stats in the world may not answer the fundamental question of "Why?" "Why did theystop visiting your site?" "Why are they spending less time there?" Or, more critically, "Why are they buyingless?" So, an understanding of the nature of your relationship with your audiences will allow you to addressthe cause of behavioral problems you identify.

Step 4: Define your benchmarks

If I tell you that 35% of the conversations about you are positive, is that a good thing or a bad thing? It couldbe that 65% of the audience DOESN’T care. Or that you just tripled your percentage of positive conversationfrom 11% the year before. If only 15% of the conversations about your competitors are positive, 35% is agreat number. But if you peers are getting scores in the 60% range and you are at 35%, then you have aproblem.Since measurement is a comparative tool, you need to figure out what you are benchmarking your results

against. It could be a peer organization, or a competitive organization, or whatever keeps your top brass or C-suite up at night. Whether you select competing organizations or peer organizations, try to limit the numberof entities in any given study to no more than five. Three is ideal, and anything more than five becomesunwieldy.



Step 5: Select a tool

In the world of social media there’s no shortage of tools. On average, someone announces a new socialmedia measurement tool every other week. The reality is that many of these "tools" are irrelevant at best and

at worst, a waste of time and money. The tool you really need depends entirely on the goals you’ve set foryour program. Which is why we always make selecting the tool the second to last step in the process of creating a measurement program.There are essentially three tools that have to be in your measurement toolkit:

1. Web Analytics and Statistical Analysis: The tool to measure financial outcomesIf your goal is sales and can be measured by click-throughs and conversions, all you need is GoogleAnalytics. Or if you want greater functionality and sophistication, you can go with tools like Omniture,WebTrends. The key element in measuring contributions to sales is to accurately factor in the cost of thatclick-through or the cost of the lead. That means you have to factor in your time, agency time, and seniormanagement time, as well as the actual cost of the program. This financial analysis allows you todetermine the efficiency of different programs. Just remember to use unique URLs to most accurately

measure results of specific programs.For example, suppose it costs you $120 a year to set up a blog and you spend an hour a day on it. If youvalue your time at $150 an hour, your cost for the year is $54,870. If the blog generates 50 click-throughsa day or 18,250 a year, your cost per click is: $54,870/(365 x 50) = $3.00.If the objective is not as directly commercial (if, for example, you want to move people along thepurchase cycle), you can measure the number of people who click through from a blog to your site. Thepercentage of all visitors who take action or click through is a fundamental measure of success.

The most important part of web analytics is to make sure you are accurately correlating results toactivities. Using a tool like SPSS or SAS, you will need to align the timing of activities such as product

launches or announcement with the activity on your blog or web site.

2. Surveys: The Tool to measure relationships

If your goal is improved relationships, then you measure those via surveys, preferably using Drs. Jim andLaurie Grunig's relationship measurement instrument (See Appendix A). There are free survey tools outthere, such as SurveyMonkey or Zoomerang, as well as thousands of established market research firmsthat will do the job. With respect to our blog example, it would be a great idea to conduct a periodic emailsurvey of just your readers (if you have a mechanism to capture emails of people who have visited yourblog).

3. Content Analysis: The tool measure perceptions and reputation

Think of social media as one enormous focus group, with customers, prospects, employees, and potentialemployees all constantly sharing their thoughts with the world. Simply counting the volume of conversations and comments is not an adequate metric. While it might give you a good feeling to knowthat your brand is being mentioned with increasing frequency in social media, it would be highlydangerous to simply assume this to be good news. One only has to look as far as Edelman, Wal-mart,



Dell, and Domino's Pizza to know that quantity of conversation may not be a measure of success.Therefore, determine the quality as well as the quantity of the discussion about your brand with athorough content analysis.Content analysis of social media gives you the opportunity to listen in on conversations. As a result youwill gain a much better understanding of how your audiences are responding to your initiatives. The

words and thoughts shared in the actual social media conversations are an important source of information, and you should be looking for messages and themes to determine how your customers andconstituencies perceive your organization or brand. How does the conversation position your brand onissues like employer of choice, value, or customer service? A good analysis will pull out recurringthemes, complaints, and messages and quantify them to determine if they require action or can beignored.If you are dealing with a blog, you also need to determine whether or not the comments are in agreementor disagreement with the blogger, and what both sides are saying. In essence, each comment needs to beanalyzed in the context of the original posting to determine if the position is widely held or just anisolated opinion. Positive or neutral comments are indicative of a healthy relationship between theblogger and his or her audience.

Additionally, social media offers the opportunity to listen in on what your marketplace thinks of yourbrand. While it may not meet the strict criteria of market research, an extensive listening program(essentially a detailed content analysis), is in essence an on-line focus group that can provide continuousfeedback on your brand and your product.

How to conduct a social media content analysis.:

1. Getting the content

Perhaps the most difficult part of any measurement program is collecting the content. All monitoring

services have their own spiders and search engines, but none of them offer any guarantee as to what theycollect. There are hundreds of millions of social media conversations going on, so you need to carefullyevaluate the claims of the various monitoring services.

There are a host of free monitoring services including Google Alerts and Social Mention. Additionallythere are monitoring systems for specific forms of social media, such as Twazzup and Tweetdeck that arespecific to Twitter.The specific words shared in social media are an important source of outtake information. Contentanalysis of blogs should look for messages and themes to determine how your customers andconstituencies perceive your organization or brand. How does social media position your brand on issueslike employer of choice, value, or customer service? A good analysis will pull out recurring themes,complaints, and messages and quantify them to determine if they require action or can be ignored.

Content analysis of social media should include an analysis of the following elements. Remember thatyou should be coding competitors' items for the same criteria you code for your own.



2. Determine the type of conversation taking place

Categorizing conversations by topic is useful, because you can then focus on certain conversations andignore others. Looking at the type of conversations that people become engaged in (i.e. comment on) can

quickly tell you the nature of the conversations. By quantifying the percentage of conversation that fallsinto each category, you efficiently decide best practices in responding to or ignoring differentconversations. Based on sociology and communications theory, and on our own empirical research andobservation, there are in life and in social media just 27 basic types of conversation:1. Acknowledging receipt of information2. Advertising something3. Answering a question4. Asking a question5. Augmenting a previous post6. Calling for action7. Disclosing personal information8. Distributing media9. Expressing agreement10. Expressing criticism11. Expressing support12. Expressing surprise13. Giving a heads-up

14. Responding to criticism15. Giving a shout-out16. Making a joke17. Making a suggestion18. Making an observation19. Offering a greeting20. Offering an opinion21. Putting out a "wanted for free" ad22. Rallying support23. Recruiting people24. Showing dismay25. Soliciting comments26. Soliciting help27. Starting a poll

If you are analyzing YouTube and other videos, you may also want to categorize them as one of thefollowing:

1. Advertisement2. Animation3. Demonstration4. Event/Performance5. Fiction6. Film7. Home Video8. Instructional Video9. Interview

10. Lecture11. Montage12. Music Video13. News Broadcast14. Promotional Video15. Sightseeing/Tour16. Slide show17. Speech18. Television Show19. Video Log

3. Determine the visibility of your brand

A key element in determining whether the discussion about your brand will be remembered and/or passedon is the visibility of your brand. Therefore you need to determine where in the post or video orconversation was your organization mentioned? Was it the focal point, or was it mentioned only in



passing? Is the post entirely about your brand or only partially about the brand. Here are some standardguidelinesDominance: The extent to which a company or brand is the focus of an item. Dominance is broken downinto four categories:

1. Exclusive: The company is the only one mentioned in the item2. Dominant: The company is the main focus of the item, but not the only company mentioned3. Average: The mention of the company is one of many integral parts of the story or is equal to

other parts.4. Minimal: No one would miss it if the mention of the company were gone.

Additionally we can also further define the extent to which your brand is going to be remembered as partof the discussion by classifying the item in terms of prominence, i.e. The location of the first mention of the company within an item. The more prominent a company name is in an item, the more likely aconsumer is to remember it. Prominence is measured differently for print items and video items.

• Print item Prominence:

1. Photo – The company name is first found in a photo or in a photo caption2. Headline – The company is first mentioned in the headline3. Top 20% - The company is first mentioned in the top 20% of the item body.4. Bottom 80% - The company is first mentioned in the bottom 80% of the item body.

• Video item Prominence:

1. Video Start – The company is first mentioned or shown at the start of the video (first 1% to 25%.2. Video Middle – The company is first mentioned in the middle of the video; (26% to 75%)3. Video End – The company is first mentioned at the end of the video (76% to 100%)

4. Determine who, if anyone, was quoted in the item?

If you are conducting a thought leadership program, tracking the presence or absence of your thoughtleaders is critical. Therefore you need to determine where any of your leadership team or thought leadersare mentioned and if they are mentioned did they convey a key message or positioning point?

5. Determine tone, sentiment, and/or presence or absence of recommendations commendation

Sentiment scoring in a content analysis can be seen as a substitute for a “Net Promoter Score” – in

essence you are determining whether consumers are recommending your brand. However, contentanalysis is also an issue of some debate, because many companies claiming they can automatically codefor sentiment. In our studies we find them to be about 60% accurate. For instance, when a person fromNew England in the U.S. says that something is “wicked good” a computer will no doubt consider thatnegative. Similarly if, in a movie review I say “saw the movie. Read the book” a computer will assume itis a recommendation for the book, not a panning of the movie. Therefore, we strongly recommend humancoding for sentiment. Typically tone should be coded on a 4 point scale as follows:



1. Favorable or positive tone is defined as “leaving the reader more likely to dobusiness/join/support the organization.”

2. Negative items leave the reader less likely to do business with the organization.3. Balanced items have both negative and positive sentiment expressed4. Neutral items convey no sentiment at all.

6. What, if any, messages were communicated?

Assuming your organization has key messages it would like the marketplace to know, you need todetermine if they are in fact being picked up by consumers. Since it is highly unlikely that consumers willpick up your messages in their entirety, you need to categorize any discussion of your messages asfollows:

1. Amplified message -- Writer communicates your key message and builds on it or amplifies itwith photos, description or recommendation

2. Full message communicated -- All aspects of the message were included3. Partial message communicated – Only part of your message was included4. No message communicated5. Incorrect message communicated6. Negative message communicated (writer says the opposite of your desired message)

7. Positioning on key issues or battles:

How was the organization (and its peers) positioned on key issues such as “good value for the money,”“effective advocate for the industry,” or “responsiveness?” Are you mentioned as an employer of choice,

and/or as delivering good value? A good analysis will pull out recurring themes, complaints, andmessages and quantify them to determine if they require action or can be ignored.

8. Quantifying the Authority of the writer or poster.

Ever since the first A-list blogger was crowned and Technorati put out its rankings, PR people have beenbegging for a way to measure authority. The most common practice is to count the number of links andcomments, and roll them up into a ranking or authority index. The simplest thing to do is to look up theURL of the blog on Technorati and see what the rank is. Be aware that Technorati's ranking have receivedsome criticism, and they may not offer rankings in the future. A step up from that is a nifty littleTechnorati-based widget on the Kineda site that will tell you instantly if a given URL is an A-, B-, C-, orD-list blog (www.kineda.com/?p=1166). There are also a number of businesses such as HubSpot(www.hubspot.com ) that have created products or services around the need for measuring authority.Another approach is to look at the specific industry and/or market and design an authority index aroundyour particular business or market. This requires more in-depth bespoke research up front, but will yieldmore useful results in the long term.



9. Defining/categorizing the author:

In traditional media you may have reporters, editors, producers and commenters. Similarly, social mediacontributors can also be classified according to the role they play in social media: Social mediaclassifications are not mutually exclusive and can vary by media usage. i.e., a person can be many types

of users at the same time. Someone may create original content on their own blog, making them a creator.That same person may be classified a respondent on other blogs when they comment on subjects. Also,that person may be a participant on Facebook (they have an account and profile), but maybe they onlyview content on MySpace and have no account, which would make them a spectator. Someone can evenbe a creator and publisher even within the same post. Someone creates and original post and as a separatetopic, part of it contains someone’s post, then that person would be a creator and publisher.

We recommend using the following definitions:

CREATOR: Creators make social content go. A social media creator is someone that producesand uploads their own original social media content. The creation can be in any format, whether itbe text, audio, video, photo or any other medium. If a person is listed as an author or creator of social media content and there is no reason to believe they are not the content maker, then theyare classified as a creator.

COLLECTOR: Collectors organize content for themselves or others using RSS feeds, tags, andvoting sites like Digg.com There are two types of social media collectors, first, someone thatgathers and organizes their social media content through RSS feeds, social bookmarking sites,and/or aggregator sites. Or, someone that uses social media sites to manage their online content,e.g., bookmarks, favorites, etc.A person can be classified as a collector by signing up on a bookmarking site or signing up for

RSS feeds to social media content.

PUBLISHER: A social media publisher uploads someone else’s content onto a social mediaplatform. They take someone else’s creation and upload it onto a social media platform. Thisincludes uploading someone else’s content, copying and pasting, forwarding, retweeting, or othersimilar practices. This can be taking content from traditional media and formatting it to socialmedia or taking content from the Internet or social media sites. Publishers are identified by thesources they site or other identifiers that signal content reproduction. If a blog post sites that thepost comes from another source, or if it is a twitter retweet, or any other indication of contentreproduction.

RESPONDENT: A respondent is someone that responds to social media content by commentingon blogs, replying to forum threads, rating videos or responding in some way to social mediacontent. Respondents can communicate positively, negatively, or neutrally. A respondent isdifferent from a publisher because they are directly responding to previously posted social mediacontent. You can be a respondent and not be a participant on some sites. Some sites only allowcomments from people with profiles, while others allow comments by anyone. Respondents areidentified simply by their commenting, rating, or replying to some other social media content.



PARTICIPANT: A social media participant is someone that is officially signed up with an officiallogin and user profile on a social media site. Examples are people signed up on Facebook,LinkedIn, MySpace, Twitter or other similar sites. Having participant status can vary by mediatype. A person may have a MySpace account and be considered a participant, while at the same

time they look at Twitter, but are not signed up, making them a Spectator. Only people withactive profiles are considered participants. A profile is deemed active if it has been used in at leastthe past six months.

SPECTATOR: Social media spectators are people that browse at content on social media sites, butare not officially logged in or signed up on that site. In cases where a user has an account e.g.,YouTube, but they view content without being signed in, they will be considered spectators. Aperson is only considered a participant when they are viewing content and are signed in with theirofficial profile. To track this metric would require analytics that could show you all of the peoplethat visit a social media site and then tell you whether they are logged in and participating or if they are not logged in and are sampling browsing.

Step 6: Analyze results, make recommendations, do it again

The most important element in any measurement system is the analysis of the data. So what if there’s a bigspike in positive conversation in June – the question is, what caused the spike? Ideally you would use a goodstatistical analysis package such as SPSS or SAS to determine correlations between conversations and webtraffic or other outcomes. At the very least, you will want to determine what isn’t working and what is. Lookfirst for weaknesses, what didn’t work. Then look for exceptional successes. Remember to analyze thecompetitive results as well. Where is the competition leading, where are you winning? What do you do with

the data once you have it? The most important part of any measurement program is teasing insight from thedata and drawing actionable conclusions.

Then look at trends over time. What happened yesterday or last week is important, of course, but what youneed to do is to see if complaints are going up or down over time, or if your relationships are getting better orworse over time, or if the ranks of complainers are growing faster than the ranks of supporters.

About negative comments: Do not, I repeat, DO NOT go into crisis mode the first time you see negativecomments. Do a bit of research first. Read the blogger's prior postings. See how many links and commentshe or she has. If it's one or two, don't do anything, but watch the numbers, if they start to grow quickly, youmay have an emerging crisis. If it's already in the hundreds, and/or if this blog is one of the top most

influential blogs in your industry, then you need to come up with a response. If it’s someone who has writtenhundreds of postings, received zero comments and does not appear on anyone's rankings, then you canprobably ignore it.

If it's not a crisis, but there is someone who is consistently writing negatively about you, take the wait-and-see attitude. Evaluate the bloggers authority. See what kinds of comments are made, and how the bloggerresponds. Then start a dialog. Offer information, a perspective or new information.



Finally, keep close track of the activity to your website and correlate that activity against the various postingsin the social media that mention your brand.

A final word on ROIThe ultimate question for social media mavens is of course, what’s the ROI. In some respects, ROI is deceptivelysimple. “R” means the return you expect, “I” is the investment you have made, i.e. staff time+agencytime+executive time+ hard costs. Subtract the I from the R and you have ROI.

And, while pulling together the costs is relatively straightforward, calculating the actual revenue is anything but.We see five ways to determine the “return.”

1. Sales – This is obviously the most direct and simple definition of ROI. If you are like Southwest airlines,you can use your web analytics and Customer Relationship Management (CRM) to tie actual sales tosocial media activity. This works well if you are selling tickets, or are managing an online sales site. Italso works well for non-profits where the goal is to increase donations.

2. Cost Savings – More and more organizations are using social media for recruitment or customer service.By carefully listening to the conversation they are able to identify better candidates, or more quickly solvecustomer problems. For example, food service company Sodexo calculated a $300,000 savings inrecruitment costs after it began using Twitter to aid recruitment..

3. Paid vs. earned search rankings – Most companies are spending significant funds on services such asGoogle AdWords to ensure that when customers search for their products, their brand shows up high upin search results. One of the benefits of a proactive social media campaign is that it makes your brandmore current and more relevant – two criteria that determines how high up in a search your brand will

appear. Many companies that have active social media campaigns find that they can drop their paidsearch ads altogether and still receive very high rankings in search.

4. Cost avoidance – Any brand has the potential to be caught in a crisis. Whatever the nature of that crisis,good relationships can help mitigate its extent and duration. The best way to calculate the value of thatmitigation is to examine what happens to competing organizations in your marketplace. Compare themoney spent on legal fees for a crisis with social media to one handled without the benefit of socialmedia.

5. Social Capital -- There has been extensive research done by Robert Putnam and others on the value of social capital and social networks. His work shows that, for an individual, the more relationships youhave, the better your life is, the longer you live, and the healthier you are. For a company, good socialcapital means that information flows more easily, innovation and efficiency increase, and legal costs godown. You can extend this concept to social media. If a blog is generating favorable comments, engagingemployees or customers in the business of the organization, and disseminating information quickly andaccurately, it is contributing to the social capital of your organization. In this case you will need tools toboth monitor and analyze the conversations, as well as data from other parts of your organization onbusiness processes such as customer turnover rates, employee turnover rates, legal costs and productivityscores (typically measured in revenue per employee).



Glossary of Measurement Terms

Over time, researchers and measurement experts have developed their own terms that can frequently be confusingto the person who is new to the measurement process. This Glossary has been adapted from the Institute forPublic Relations Dictionary of Measurement Terms, edited and compiled by Dr. Donald Stacks of the Universityof Miami, and fully downloadable from the IPR’s website, www.instituteforpr.org. Italic terms in the definitionsindicate cross references.

Baseline or benchmark: An initial measurement against which subsequent measures are compared.Benchmarking or benchmark study: measurement technique that involves having an organization learnsomething about its own practices and/or the practices of selected others, and then compares these practices.Research that establishes a benchmark.AdWords: Google's program of pay-per-click search page advertising.

Authority: The ability of a blog to command respect, and to influence, thought, opinion, or behavior.Blog: Short for "Web log," an online journal generally updated very often. In order to be considered a blog, thesite must allow comments.Click-through: An online behavior metric representing an individual responding to an ad or link by clicking onit.Circulation: Number of copies of a publication as distributed. Not usually the same as the number actually read,but as a practical matter, synonymous with opportunities to see, impressions, and reach.Column inches: The total length of a printed article if it were all one column, measured in inches.Content analysis: An informal research methodology and measurement tool that systematically tracks messages(written, spoken, broadcast) and translates them into quantifiable form by defining message categories and

specified units of analysis.Conversion: When a prospective customer takes the marketer's intended action. For a website, a conversionmight be making an online purchase, or submitting a form to request additional information. The conversion rateis the percentage of visitors who take the conversion action.

Dashboard: A technique for simplifying data reporting by displaying a small number of important summarymeasures together in one location. Like an automotive dashboard, a PR dashboard includes only those measuresmost critical for assessing the progress or health of a program or company.editorial or earned media -- (1) The content of a publication written by a journalist, as distinct from advertisingcontent, which is determined by an advertiser; (2) An article expressing the editorial policy of a publication on amatter of interest (also known as a Leader or Leading article); (3) Space in a publication bought by an advertiserthat includes journalistic copy intended to make the reader think it originates from an independent source (also

known as an Advertorial).Engagement: Meaningful connection or interaction between a consumer and a website, organization, product, orbrand.Focus group methodology: Informal research technique that uses a group discussion approach to gain an in-depth understanding of a client, object, or product; is not generalizable to other focus groups or populations.Frequency -- descriptive statistic that represents the number of objects being counted (for example, number of advertisements, number of people who attend an event, number of media release pickups).



Impressions: Opportunities to see an article or message generated by the total audited circulation of apublication. For example, if The Wall Street Journal has an audited circulation of 1.5 million, one article in thatpaper generates 1.5 million impressions or opportunities to see the story. Two articles generate 3 millionimpressions, and so on. Opportunities to see, circulation, impressions, and reach are synonymous.Key Message: Specific statement or concept that an organization is trying to communicate about itself. A

common general goal of PR is to get key messages into media coverage. A key message ought to be unique toyour organization and it must be something that a journalist is likely to print, for example, Company X providesthe best customer service in the industry or, Company YÕs product is of the highest quality.Message content: (1) The verbal, visual, and audio elements of a message; (2) The material from which contentanalyses are conducted; (3) A trend analysis factor that measures what planned messages are actually contained inthe media.Message content analysis: -- Analysis of media coverage of a client, product, or topic on key issues.

Omnibus survey -- An all-purpose national consumer poll usually conducted on a regular schedule (once a weekor every other week) by major market research firms; also called a Piggyback or shared-cost survey.Opportunities to see (OTS): A number equal to the total audited circulation of a publication. Opportunities tosee, circulation, impressions, and reach are synonymous.

Outcomes: Quantifiable changes in attitudes, behaviors, or opinions that occur as end results of a PR program.Outputs: The physical products of a PR program; anything that is published or directly produced by the publicrelations team. Outputs can be articles, white papers, speaking engagements, the number of times a spokespersonis quoted, specific messages communicated, specific positioning on an important issue, or any number of quantifiable items.Outtakes: What members of your target audiences take away from your program - the messages, perceptions andunderstandings that your program has generated. Outtakes are the perceptions generated by your outputs.Poll: (1) A form of survey research that focuses more on immediate behavior than attitudes; (2) A very shortsurvey-like method using a questionnaire that asks only very short and closed-ended questions.Positioning: How an organization is perceived on broad industry characteristics, such as leadership, innovation,

employer of choice, neighbor of choice, and so forth.Program or campaign: The planning, execution, and evaluation of a public relations plan of action aimed atsolving a problem.Public: (1) A group of people whose behavior may have consequences for an organization or who are affected bythe consequences of organizational decisions; (2) A group of people from which a public relations campaign orprogram selects specific targeted audiences in an attempt to influence behavior or attitudes regarding a company,product, issue, or individual.Qualitative research: Studies that are somewhat to totally subjective, but nevertheless in-depth, using a probing,open-ended response format.Quantitative research: Studies that are highly objective and projectable, using closed-ended, forced-choicequestionnaires; research that relies heavily on statistics and numerical measures.Questionnaire: Measurement instrument that uses questions to collect data for the analysis of some aspect of agroup. May be employed through the mail, Internet, in person, or via the telephone. May be both closed-endedand open-ended, but typically employs more closed-ended questions. A questionnaire is the instrument used in asurvey.Reach: The scope or range of distribution and thus coverage that a given communication product has in atargeted audience group. The total audited circulation of a publication. In broadcasting, the net unduplicated (alsocalled deduplicated) radio or TV audience for programs. Opportunities to see, circulation, impressions, and reach



are synonymous.Reach demographics: Reach into specific demographic segments, determined using data from one of thegenerally accepted sources such as SRDS or Simmons.

Twitter: An Internet micro blogging service that restricts messages to 140 characters.Sample: A group of people or objects chosen from a larger population.

Survey: The process of gathering data from a sample of a population. The instrument used in a survey is called aquestionnaire.

Target audience: Specific subset of a total audience, differentiated by some characteristic or attribute (forexample, sports fishermen), that is the specific focus of a marketing or public relations effort.Tone: Trend and content analysis factor that measures how a target audience feels about the client, product, ortopic; typically defined as positive, neutral/balanced, or negative.Trend analysis: Tracking of performance over the course of a PR campaign or program. A survey methodwhereby a topic or subject is examined over a period of time through repeated surveys of independently selectedsamples.Viral: Spread rapidly by social media or word-of-mouth, creating buzz and interest.



Bibliography and Measurement Resources

Grunig, J.E., & Grunig, L.A. (1992). Models of public relations and communications. In J.E. Grunig (Ed.),Excellence in public relations and communications management. Hillsdale, NJ: Lawrence Erlbaum.

www.themeasurementstandard.com KDPaine & Partners - monthly newsletter on public relations measurement.http://kdpaine.blogs.com/ Katie Delahaye Paine's public relations measurement blog.www.instituteforpr.org the site of the Institute for Public Relations; includes many free papers and resources.



The Grunig Relationship Index:

Questions to measure trust, including the dimensions of integrity, competence,dependability

1. This organization treats people like me fairly and justly. (Integrity)2. Whenever this organization makes an important decision, I know it will be concerned about people

like me. (Integrity; original dimension: faith).3. This organization can be relied on to keep its promises. (Dependability)4. I believe that this organization takes the opinions of people like me into account when making

decisions. (Dependability)5. I feel very confident about this organization’s skills. (Competence)6. This organization has the ability to accomplish what it says it will do. (Competence)7. Sound principles seem to guide this organization’s behavior. (Integrity)8. This organization does not mislead people like me. (Integrity)9. I am very willing to let this organization make decisions for people like me. (Dependability)10. I think it is important to watch this organization closely so that it does not take advantage of people

like me. (Dependability) (Reversed)11. This organization is known to be successful at the things it tries to do. (Competence)

Questions to measure Control Mutuality

1. This organization and people like me are attentive to what each other say.2. This organization believes the opinions of people like me are legitimate.3. In dealing with people like me, this organization has a tendency to throw its weight around.

(Reversed)4. This organization really listens to what people like me have to say.5. The management of this organization gives people like me enough say in the decision-making

process.6. When I have an opportunity to interact with this organization, I feel that I have some sense of control

over the situation.7. This organization won’t cooperate with people like me. (Reversed)8. I believe people like me have influence on the decision-makers of this organization.

Commitment

1. I feel that this organization is trying to maintain a long-term commitment to people like me.2. I can see that this organization wants to maintain a relationship with people like me.3. There is a long-lasting bond between this organization and people like me.4. Compared to other organizations, I value my relationship with this organization more.5. I would rather work together with this organization than not.6. I have no desire to have a relationship with this organization. (Reversed)



7. I feel a sense of loyalty to this organization.8. I could not care less about this organization. (Reversed)

Satisfaction:

1. I am happy with this organization.2. Both the organization and people like me benefit from the relationship.3. Most people like me are happy in their interactions with this organization.4. Generally speaking, I am pleased with the relationship this organization has established with people

like me.5. Most people enjoy dealing with this organization.6. The organization fails to satisfy the needs of people like me. (Reversed)7. I feel people like me are important to this organization.8. In general, I believe that nothing of value has been accomplished between this organization and

people like me. (Reversed)

Communal Relationships

1. This organization does not especially enjoy giving others aid. (Reversed)2. This organization is very concerned about the welfare of people like me.3. I feel that this organization takes advantage of people who are vulnerable. (Reversed)4. I think that this organization succeeds by stepping on other people. (Reversed)5. This organization helps people like me without expecting anything in return.6. I don’t consider this to be a particularly helpful organization. (Reversed)7. I feel that this organization tries to get the upper hand. (Reversed)

Exchange Relationships

1. Whenever this organization gives or offers something to people like me, it generally expectssomething in return.

2. Even though people like me have had a relationship with this organization for a long time, it stillexpects something in return whenever it offers us a favor.

3. This organization will compromise with people like me when it knows that it will gain something.4. This organization takes care of people who are likely to reward the organization.



About the Author

Katie Delahaye Paine is the founder and CEO of KDPaine & Partners, LLC. In the relentless pursuit of

quantitative and qualitative measures of success, she and her employees have read and analyzed millions of newsarticles, blogs, newsgroup postings, and internal communications, and have conducted hundreds of thousands of interviews. Most recently her endeavors have been focused on social media measurement as well as providingcost-effective measurement programs for nonprofits, small businesses, and government agencies.Katie is a founder and former chair of the Institute for Public Relations Special Commission on Measurement andEvaluation. She served as the U.S. liaison to the European Standards Task Force to set international standards formedia evaluation. She writes a regular column for PR News on corporate image and crisis communications, andcontributes to Communications World, PR Week, Business Marketing, and New Hampshire Magazine.An accomplished speaker, Katie frequently lectures to conferences and universities including The AmericanStrategic Management Institute, the Public Relations Society of America, the International Association of

Business Communicators, and the Institute for International Research. Katie began and still writes the worldÕsfirst measurement blog (http:// kdpaine.blogs.com) and publishes The Measurement Standard newsletter(www.themeasurementstandard.com).

kdpainesocialmediawhitepaper

Documents