Hagan School of Business

Data Collection

Once organizations have been chosen to analyze and benchmark, the data collection can begin. The data can be gathered internally, externally, or through original research. Most benchmarking data collection begins with internal data collection, then proceeds to publicly available secondary source data, and finally to competitive benchmarking with external institutions. Table 1 lists potential data and information sources for higher education based on the following three categories.

Table 1 - Information Sources adapted from (Camp, 1989)
 

Source	Example
Internal
Library data bases	AB information, ABI Inform
Internal reviews	Internal experts
Internal publications	Varies by organization
External
Professional associations	American Marketing Association
Industry publications	Electronic Business
Special industry reports	ADL Infotran
Functional trade publications	Materials Handling Engineering
General management	Industry Week
Seminars	By professional interest
Industry data firms	APQC, TBE, K.L. Worthington
Software/hardware vendors	DEC, SCT, TRG
University sources	By profession
Advertisements	By product of intereat
Newsletters	By subject matter
Original Research
Customer feedback	Focus groups
Telephone surveys	Specific design
Inquiry service	Specific contract
Networks	Electronic, internal, and external
Consulting Firms	McKinsey

Internal information can be obtained from a variety of sources, and has been shown to be productive because it is cost-effective and can uncover additional leads to pursue. A product or service analysis of the process from another organization is a good first step. Ordering a catalog or brochure, requesting information, and analyzing what is received can reveal valuable customer assistance information. Often, another place to uncover data about the organization being studied is often right inside the home institution. At first it may seem unlikely that the company interested in obtaining the data may already have some useful information. Frequently, employees are keen observers of other organizations or may even have been previously employed at the institution being researched. Seeking out their knowledge and assistance would be wise (Camp, 1989).

The second category of public domain information includes many of the sources we have already seen, such as journals, magazines, and electronic databases. The company should not be overlooked as an important source for benchmarking data. In addition, library searches today can be very fruitful with the use of electronic reference searches. Professional associations, consultants, and external experts such as those discussed previously can also yield data for no or reasonable costs. Once the categories of internal and external investigations for data have been completed, there is no choice but to proceed right to the source of the best practice leader identified in the first step.

Original research can be somewhat more expensive than other approaches for data collection we have seen, but the results can be very rewarding. Just as in other academic or business research, it is best to approach a prospective benchmarking partner with a plan for collecting the data. A questionnaire can be useful for listing the data to be benchmarked, and permits more extensive data gathering. Questionnaires can be completed in several ways, including mail, telephone, and in-person (Camp, 1989). In the Benchmarking Workbook, by Gregory H. Watson (1992), many useful forms are listed, including a benchmarking questionnaire proposed for use in a study of original research. The suggested questionnaire lists the following questions to be made when contacting another organization:

1. How do you define the process? Please describe it.

2. Do you consider this process to be a problem or concern in your company? If not today, was it a problem in the past?

3 What is the measure of quality for this process? What are the criteria that you use to define excellence in process performance? How do you measure the output quality of this process? How do you measure progress in quality improvement?

4. How do you consider cost and schedule in this process?

5. How much and what type of training do you provide for the various job categories of the process team?

6. What process improvements have given you the best return in performance improvements?

7. What company, excluding your own, do you believe is the best in performing this process? (Watson, 1992)

As in most benchmarking projects, initial contacts showed that most institutions were willing to do the considerable work necessary to complete the form in order to receive a copy of the resulting data. Real benchmarking involves reciprocity, creating a "win-win" situation of information exchange that is mutually beneficial (Watson, 1993).

Camp, R. C. (1989). Benchmarking: The Search for Industry Best Practices That Lead to

Superior Performance. Milwaukee: ASQC Quality Press.

Watson, G. H. (1992). The Benchmarking Workbook: Adapting Best Practices for

Performance Improvement. Portland, Oregon: Productivity Press.

Watson, G. H. (1993). Strategic Benchmarking: How To Rate Your Company's

Performance Against the World's Best. New York: John Wiley and Sons.