Does an Online Directory Get Used, and Does Use Become Revenue? A Keep-or-Stop Walk-Through on Five Sources About Other Directories

A publications or membership manager is asked whether to keep paying to produce and host an online buyers guide or member directory, and two questions have to be answered in order: whether anyone uses it, and whether that use turns into money. We read five documents on other directories, and none of them describes a buyers guide or supplier directory that an association publishes, and none measured what a directory earns for the organization that runs it. One is an association’s survey of its own members from 2011. The others cover a wedding-service website in 2006, a restaurant-search app in Asian cities in 2014, a university’s public researcher profiles in 2012 and health insurers’ provider directories in 2018. Each section below takes one question in the order a manager would ask it, says what a source measured, and ends with the choice it leaves to you.

First, say what “used” means

A 2011 member survey from the American Association of Physics Teachers, prepared by the association’s communications and membership director, went to 8,732 members and was completed by 1,049, about 12 percent, so it describes the members who answered. Asked how important the association’s services were, 22.7 percent rated the membership directory not important. Among the 873 who rated parts of the association’s website they had visited in the past year, 43.5 percent chose “didn’t look for this” for the member directory, against 9.2 percent for membership renewal and 49.5 percent for the career center. The survey measured opinion and self-reported website use, not visits, leads or revenue.

A 2014 article in the Journal of Medical Internet Research, by authors mostly at the University of California, San Francisco, counted visits to that university’s public researcher profiles over 2.5 years of web analytics. In the final six months the site averaged 46,000 visits a month, and 81.15 percent of visits in that period viewed one page. Over 16 months, a feedback form drew 469 submissions among 670,000 visits, or 0.07 percent. The authors say they cannot conclude how often visitors who explored further achieved their goal.

To decide: write one line naming what counts as use of your directory: a visit, a click to a listing, or a contact that the listed company can confirm. These sources counted visits, clicks, calls and self-reports, and none counted a sale made through a directory.

Next, find out how people arrive

A 2011 article in Management Science, from two authors at the MIT business school, reports a field experiment on a website listing wedding services in one New England state. The site’s aggregate statistics suggest that over three-quarters of visitors to the list page came directly from search engines, and about 40 percent of list-page visitors clicked no listing. In the university profile study, 74.78 percent of visits (208,304 of 278,570) came through a public search engine, and in a sample of 250 searches that reached the profiles, 210 included a person’s name. The authors tie that search traffic to a deliberate effort to make the pages easy for search engines to read.

We read both as describing sites that most visitors reached by way of a search engine, and neither tested how traffic would change if the site were harder to find.

To decide: search a public search engine for the names of a few listed companies and note whether the directory page appears. If the guide sits behind a login or inside a PDF, say so in the memo, because these two sites were reached mostly that way.

Then look at what the listing shows and where it sits

The wedding-service list page showed only a business’s name, location and telephone number. Over two months in 2006 the site tested alphabetical order against ranking by past clicks, with and without the click counts displayed, assigning the conditions at random by category. The authors read the comparison as suggesting that some, but not all, of the benefit earlier popularity brought came through the premium positions that popular listings tend to hold. They add that the outcome was clicks and not purchases, and that the data hold one experiment in one product market and one area.

To decide: record the rule that orders your entries and what each one displays, since the one test here that tried to separate position from popularity found that position carried part of the effect.

Paid listings are a separate question

A 2020 article in the Review of Economic Studies, by two Stanford authors, reports an experiment inside one restaurant-search app, in which about 200,000 users in 13 Asian cities were randomized into seeing restaurant ads. Some of them were told that a listing was a paid ad and the others were not, with the ads otherwise the same. The authors report that telling users raised calls to the restaurant by 77 percent. The base rates were small: calls followed 0.031 percent of first ad exposures without the label and 0.055 percent with it, and a footnote counts 23 calls in one group and 161 in the other, the second group about four times larger. Page visits did not differ significantly between the two at the 95 percent confidence level, the outcome was calls because visits and spending could not be tracked, and advertisers received the ads free during the test.

This is one restaurant app in 2014, outside the United States, and it tested a label on a listing, not a price, a placement or a renewal. It says nothing about what a directory earns from selling listings.

To decide: if your listings are sold, write down whether they are marked as paid and what the buyer is promised, and keep any claim about their effect to what you have measured yourself.

Last, ask how wrong the entries are

A 2020 article in Health Affairs, by authors at Yale and New York University, reports a survey of privately insured US adults in August and September 2018 about their insurers’ directories of mental health providers. Of 837 who had used outpatient mental health care, 44 percent had used the plan’s directory in the past year, and 53 percent of those users had met at least one error. The most common was a provider wrongly listed as taking new patients, at 36 percent. Users who met an error were more likely to be treated out of network, at 40 percent against 20 percent, and to receive a surprise outpatient bill, at 16 percent against 4 percent. The authors say the data cannot show whether more accurate directories would mean fewer out-of-network patients, and that people treated out of network may have been more likely to recall an error.

These are insurers’ directories used to choose a doctor, not an association’s. The map sheet covers who checks entries; here the point is narrower. In our reading, a low use count cannot tell a directory nobody wants from one that people stopped trusting, and none of the five separated the two.

What the memo can claim

Report use as one count with its definition, name the share of entries someone confirmed this year, and say that how often a visit turns into a purchase is unmeasured. Four of the five sources describe data from 2014 or earlier, two were run on a company’s website or app, and none involved a buyers guide or supplier directory. The suggestions in this walk-through are our reading, and none of the five sources tested them.

These five sources do not show that a paid association guide earns revenue, how many buyers or members open one, or what level of accuracy changes use.