Georgia Tech Policy Project on Industrial Modernization
HOME | THE PROJECT | PUBLICATIONS | SEARCH | CONTACT US | WHATS NEW

GaMEP Evaluation Working Paper E9801

Jan Youtie, Georgia Tech Economic Development Institute, and Philip Shapira, School of Public Policy, Georgia Institute of Technology, September 1998.

This page contains an Executive Summary of the report and the Table of Contents. The entire report can be viewed in Adobe Acrobat format. To view report [Select *.PDF 176K].

For further information on using PDF files and installing Adobe Acrobat select [here].

This study was undertaken as part of the evaluation component of the Georgia Manufacturing Extension Partnership (GaMEP).  To view the full list of GaMEP Evaluation Working Papers select [here].

This report examines staff and participant views of manufacturing-based training programs offered by Georgia Tech's Economic Development Institute (EDI) through the Georgia Manufacturing Extension Partnership and associated EDI centers. An assessment is made of the impact and value of these training programs, particularly from the view of company participants. While most participants report beneficial impacts from participation, some recommendations for improvement are offered.

The report is based on surveys of coordinators of 124 EDI training programs and long-term follow-up interviews with 39 training course participants. Acknowledging the small number of course participants responding to our follow-up survey, the report also draws on data on the training interests and needs of Georgia manufacturers reported in the most recent Georgia manufacturing technology survey.

One of the reasons for undertaking this study was to better understand the resources that are involved in mounting manufacturing-related training and the kinds of outcomes that result. Within the economic development and manufacturing extension communities, training activities have received far less long-term evaluation than individual projects. This study develops applicable methodologies and presents baseline data that can be used for further follow-up assessments of manufacturing-related training initiatives.

The principal findings contained in the report are summarized below:

• Quality-related courses offered by EDI's Center for International Standards and Quality (CISQ) accounted for more than half of EDI's 1997 course offerings. Information Technology courses offered by EDI's Center for Manufacturing Information Technology (CMIT) accounted for nearly 30 percent of 1997 course offerings.
• EDI’s portfolio of training programs was compared with what the Georgia manufacturing survey said about manufacturer interests in training. We found that EDI offered more quality-related courses, roughly an equivalent amount of information technology and lean manufacturing courses, and fewer courses in human resources, energy and environmental management, and marketing and business management.
• Attracting companies to undertake follow-on technical assistance projects, generating fee revenue, fulfilling individual or office work plans, and serving a large number of firms relatively inexpensively were the most common objectives of EDI staff in offering manufacturing-related training.
• Most courses were offered before, and the average course had been previously offered four times. Information technology and lean manufacturing courses were developed and offered more recently than those in the quality area. (Quality courses are revised annually).
• About 70 percent of the courses were offered in Atlanta.
• The typical (median) course had 12 participants from six companies. In all, 1,174 participants representing 612 companies attended the training activities profiled in this report.
• To conduct these courses, nearly 1,600 EDI staff hours were involved. For every training participant, EDI staff put in 1.3 hours of work, on average. CMIT training required the most EDI staff hours per participant; CISQ training required the least staff hours per participant. Contract courses required more effort than open enrollment, conferences, or network groups.
• EDI’s out-of-pocket expenses (excluding staff costs) for training in 1997 were around $132,000 (mostly from CISQ-delivered courses). This total expense figure includes more than $56,000 returned to Georgia Tech's Distance Learning and Continuing Education Department.
• Total program revenues exceeded $393,000 and a gross program income of nearly $261,000 was reported. However, if staff hours (valued at $280 per hour) are included, the cost of EDI's manufacturing training events¾ both contract courses conducted for individual firms and open enrollment courses¾ significantly exceeded revenues. This does not necessarily mean that EDI’s investment in training is not worthwhile from a program or economic development perspective. Nor does this accounting consider the full tangible and intangible benefits and costs to the firms themselves associated with their participation in EDI training.
• Many courses involve individuals or organizations outside of Georgia Tech; the most common functions for these individuals or organizations is to make presentations or, less frequently, to support marketing.

• The most common reasons for participants taking courses were that they were requested by a supervisor or required by an employer, or they had a personal desire to learn a new skill. Certification required by the company was an important motivation for taking quality-related courses.
• Participants enrolled in the particular EDI course because it met their needs and because of Georgia Tech's reputation.
• Referrals from supervisors or co-workers, direct mail from Georgia Tech, and Georgia Tech field staff word-of-mouth were the most common ways that participants found out about the course.
• Nearly two-thirds of participants were somewhat familiar with the subject matter prior to taking the course.

• EDI staff use a broad range of customized end-of-session forms to obtain participant evaluations, making cross-course comparisons difficult without directly surveying participants. However, based on our own direct follow-up survey of participants, 62 percent of participants were very satisfied with the course overall, and nearly half said they gained a great deal of knowledge as a result of the course. Participants in quality-related courses were most likely to be very satisfied and report gaining a great deal of knowledge.
• Most participants found courses useful to their job (55 percent), used what they learned often (67 percent), and found it easy to apply what they learned to their job (59 percent). quality-related courses were more likely to be rated very useful (81 percent), to be used often (75 percent), and to be easy to apply (80 percent).
• Most participants believed that the training resulted in improvement in certain areas of operation and business performance. The average participant identified eight areas improved, including improved quality (59 percent), improved shop floor processes (51 percent), changes in business or management strategy (49 percent), and improved employee skills (46 percent). Training courses tended to have impacts in expected areas related to their subject matter.
• Only four participants were able to assign a monetary value to benefits, and 23 participants were able to monetarize costs. Quality-related courses appeared to have to highest company costs.
• Although most EDI staff do not track follow-on requests for information and projects, 41 percent of participants reported requesting technical assistance, 26 percent reported requesting additional training courses, and 13 percent reported requesting assistance from another organization to which they were introduced through the training course.
• If the course had not been offered from Georgia Tech/EDI, 21 percent that said they would have not taken the training without the Georgia Tech course, and 62 percent of respondents said they would have attended a similar course offered by another institute or vendor. Manufacturing conference attendees were somewhat more likely to suggest that they would not have taken the training without the Georgia Tech conference.

• EDI's manufacturing-related training is too concentrated in quality and information technology areas. EDI should pursue opportunities to develop course offerings in additional areas of interest to manufacturers, such as marketing, lean manufacturing, and environmental/energy. In 1998 for example, EDI developed three energy-related certificate courses. In some cases, EDI may wish to jointly sponsor these training initiatives.
• From a fee-generation standpoint, training is an important area for EDI. Training involves 12 percent of EDI's business and industry budget, but it generates nearly 40 percent of revenues. Because of its importance in fee generation, EDI should look more carefully at the value-pricing of its training programs. In particular, there may be opportunities to increase prices, although this would need to be weighed against potential declines in participation and comparisons with similar private and public sector training fees.

• Company respondents most consistently indicated that they valued EDI’s quality programs. Training in the quality area follows an effective model consisting of more intensive training, well-conceived curricula, goal-orientation, group service delivery, systematic utilization internal of resources, intensive marketing, and utilization of a cadre of effective trainers. Drawing on the positive feedback reported on EDI’s quality training programs, it may be useful to for EDI to promote more cross-disciplinary exchanges between different training areas to disseminate insights and effective practices.
• The information technology training area received less consistent indications of value from company respondents. EDI’s information technology initiatives are newer and more experimental and the training curricula change very rapidly with the current fast pace of information technology development. In addition, information technology courses have far fewer internal resources allocated to them. The information technology area should either adopt the key features of the model of training used in the quality area or explicitly run training using the minimal amount of resources necessary to attract requests for technical assistance. The loss leader approach might require simpler "overview" course offerings, less internal staff time allocated to training, and more emphasis on technical assistance service marketing.

• It is worthwhile for EDI to host its Manufacturing Conference or another major event once a year.

• In marketing EDI's manufacturing services, the Georgia Tech name is important to manufacturers. Regardless of the unit offering the course, marketing should emphasize the Georgia Tech name. In addition, marketing approaches should account for participation from more than one person in a company, the use of referrals of course information from company supervisors, and the fact that most participants have some knowledge of the subject matter prior to taking the course.

EDI needs to develop more consistent and effective procedures through which data on training events and participation (and links, where appropriate, with follow-on projects) can not only be recorded, but also retrieved and analyzed. Consistent end-of-session evaluation procedures and protocols would also aid the cross-program analysis of training initiatives.

Contents

TOP | HOME PAGE