Automotive Program Evaluation Guide

Click here to download this guide in Adobe PDF format.

Everyone associated with an automotive program, whether it is an automobile, collision repair & refinish, medium/heavy truck, alternative fuels, or general service, should be aware that an extensive self-evaluation must be conducted by school personnel and certain criteria must be met to be approved for an on-site team evaluation. Documentation must be available for the on-site team to verify that the program meets all requirements for ASE certification. The good news is that the on-site team will evaluate exactly the same items the school evaluated.

The following document was prepared by a group of experienced ETLs, NATEF staff and Trustees in March 2004. The result is a collection of suggestions for schools and ETLs alike to be used as a guide for preparing, reviewing, and evaluating the documentation needed for program certification. These suggestions are meant as examples and we are confident that there are many other documents that can be used to show how programs meet the standards for certification.

NOTE: This example was developed to correspond to 2004 and later versions of NATEF standards. If you have an earlier version of the standards, the exact wording may be different, but the ideas, procedures, and principles are the same.

To complete the Self-Evaluation Form, it is necessary to follow several steps. The following is an example using Standard 1.1 B.

1. Read the statement on the form.  It reads: “Rate the administration and use of an annual survey to determine the percentage of students who complete the program and obtain employment in the automotive industry or continue automotive education.”

2. Refer to the Program Standards section of the certification manual for additional information on each standard sub-section. Standard 1.1 B reads, “ The employment potential for automobile technicians, trained to the level for the specialty or general areas outlined in the program goals, should exist in the geographic area served by the program.”

3. Review the suggestions in this document. The suggestion for this example is for you to provide a copy of the annual survey and a summary of the results.

4. Find all appropriate reference materials, make copies of the information, clearly mark the reference or highlight specific information. Locate the annual survey of employers, make a copy of it, highlight pertinent information, and summarize the results. 

5. Determine how well your program meets the standard using the 1 - 5 rating scale, record the rating on the Self-Evaluation Form, and record the materials you used to make that rating in the Reference Materials area for 1.1. You may want to include annual surveys from the past several years to show a history of completing annual surveys and that you use the information to modify the program, if applicable.

6. Make a file for each Standard and place all Reference Materials in the file, clearly labeled according to Standard and sub-section. Some programs put everything in a 3-ring binder with sections for each standard and sub-section. Other programs prefer to make files and use file folders for each standard and sub-section. The important thing to remember is that the information is organized, labeled, and accessible for the evaluation team not familiar with your program.

7. Prior to the on-site evaluation, review all the files and make sure all Reference Materials are available for the on-site team. The more organized you are for the on-site evaluation, the easier it will be for the team to evaluate your program. The evaluation team will look at the same statements and will use the information you provide them to rate the items. (The evaluation team should make comments on any sub-section that is rated above or less than 4. These comments should be shared with the program during the debriefing meeting with school personnel at the conclusion of the evaluation.)

It is always preferable to have several people involved in the self-evaluation process. NATEF suggests that the administrators, counselors, instructors, and Advisory Committee members be involved as some of the standards evaluate their involvement in the program. It also allows for review, discussion of ideas, and plan of action by a shared group of individuals taking ownership of the program.

By the time your program has been approved for an on-site evaluation, you should be ready for the evaluation team if you have answered the questions honestly and prepared the necessary documentation. The team may ask you to clarify or provide additional information or documentation during the on-site. Remember that the team is there to verify that you meet the standards, not to find fault with your program.

STANDARD 1 – PURPOSE

THE AUTOMOTIVE TECHNICIAN TRAINING PROGRAM SHOULD HAVE CLEARLY STATED PROGRAM GOALS, RELATED TO THE NEEDS OF THE STUDENTS AND EMPLOYERS SERVED.

1.1 EMPLOYMENT POTENTIAL

1. -  B.  Provide a copy of the annual survey and a summary of the results.

1.2 PROGRAM DESCRIPTION/GOALS

1. Provide a copy of the brochure and/or catalog with appropriate pages identified (use sticky notes, highlighter, etc. to make the information easy to find).
2. Identify the location of the program materials and the distribution methods used to provide the information to students.

STANDARD 2 – ADMINISTRATION

PROGRAM ADMINISTRATION SHOULD ENSURE THAT INSTRUCTIONAL ACTIVITIES SUPPORT AND PROMOTE THE GOALS OF THE PROGRAM.

2.1 STUDENT COMPETENCY CERTIFICATION

1. Show an example of the certificate, diploma, transcript, or degree plan.

2.2 CHAIN OF COMMAND

1. Show a copy of the school organizational chart.

2.3 ADMINISTRATIVE SUPPORT

1. - D. Provide a copy of the school policy or letter of support from the administration that addresses the various issues of planned in-service and update training; tools, equipment, and service publications; curriculum; and budget preparation.

2.4 WRITTEN POLICIES

1. - C. Provide a copy of the school policy and teacher/student handbook with pages marked with sticky note and references highlighted.

2.5 ADVISORY COMMITTEE

1. - C. Provide a copy of the Advisory Committee minutes which should include the date of the meeting, who was in attendance, start and end time, agenda, topics and summary of discussion, motions for vote, actions taken, follow-up to be completed, and date of next meeting. Provide a complete list of Advisory Committee Members. For initial certification, show records for the past year. For recertification, show all records since the last certification.

2.6 PUBLIC/COMMUNITY RELATIONS

1. This is similar to 1.2 – show press releases, newspaper articles, fliers, trophies, etc.

2.7 LIVE WORK ACCOUNTING

1. - B. This applies only to programs that use customer vehicles. Show the policy statement on collecting, disbursing, and accounting for funds.

STANDARD 3 – LEARNING RESOURCES

SUPPORT MATERIAL, CONSISTENT WITH BOTH PROGRAM GOALS AND PERFORMANCE OBJECTIVES, SHOULD BE AVAILABLE TO STAFF AND STUDENTS.

3.1 SERVICE INFORMATION

1. State the location of all service information such as manuals, CDs, on-line access, etc.

3.2 MULTIMEDIA

1. Provide a list and give the location of all technology available for student and instructor use.

3.3 INSTRUCTIONAL DEVELOPMENT SERVICES

1. Indicate who provides media development services.
2. State the location of equipment and supplies.

3.4 PERIODICALS

1. Provide a list, give the location, and show examples of periodicals.

3.5 STUDENT MATERIALS

1. A. - B. Provide a copy of each textbook and other materials used for instruction.

STANDARD 4 – FINANCES

FUNDING SHOULD BE PROVIDED TO MEET PROGRAM GOALS AND PERFORMANCE OBJECTIVES.

4.1 PROGRAM TRAINING COST

1. For program budget, please only consider those expenses unique to the automotive program (e.g. consumable supplies, instructor development, curriculum development, environmental and other program contract services, amortized capital equipment, etc.) divided by full-time enrollment (FTE).

4.2 BUDGET

1. State the process used to determine the program budget.
2. Highlight pertinent discussion in Advisory Committee meeting minutes.
3. Provide budget information and highlight pertinent discussion regarding budget in Advisory Committee minutes.

4.3 BUDGET PREPARATION

1. Refer to 4.2 A. Provide copies of budget requests. ETLs may interview program staff.

4.4 STATUS REPORTS

1. Provide a copy of the last quarter’s report.

STANDARD 5 – STUDENT SERVICES

SYSTEMATIC SKILLS ASSESSMENT, INTERVIEWS, COUNSELING SERVICES, PLACEMENT, AND FOLLOW-UP PROCEDURES SHOULD BE USED.

5.1 SKILLS ASSESSMENT

1. Provide the policy statement and a description of the process used for skills assessment. Skills assessment may take place prior to or early in the program. Provide a copy of the assessment instrument, if available.
2. Provide program explanatory material with pertinent information highlighted. Note availability for students.
3. Highlight pertinent information in program materials, catalog, brochure, etc.

5.2 PRE-ADMISSION COUNSELING

1. Highlight access to the career counseling process and student services available, as sited in catalog or other materials.

5.3 STUDENT RECORDS

1. Explain how records are maintained; give their location, and length of time kept.

5.4 PLACEMENT

1. Provide the policy or explanation of the placement process.

5.5 ANNUAL FOLLOW-UP

1. - D.  Provide an explanation and a sample document.
5. Describe the procedure to use the information obtained in follow-up and give an example of changes made to program based on feedback, if available.
6. F. Highlight pertinent information in Advisory Committee minutes.

5.6 LEGAL REQUIREMENTS

1. Provide copies of Policies and Procedures. Post Haz-Mat signs. Show MSDS sheets.

STANDARD 6 – INSTRUCTION

INSTRUCTION MUST BE SYSTEMATIC AND REFLECT AUTOMOBILE PROGRAM GOALS. A TASK LIST AND SPECIFIC PERFORMANCE OBJECTIVES WITH CRITERION REFERENCED MEASURES MUST BE USED.

6.1 PROGRAM PLAN

1. Provide a copy of the course outline and brochure.

6.2 STUDENT TRAINING PLAN

1. - B. Show an example of a student training plan or advisement sheet.

6.3 PREPARATION TIME

1. Show a copy of the Master Schedule and instructor office hours.

6.4 TEACHING LOAD

1. - B.  Show student enrollment sheets, indicate the number of training stations, and identify teaching assistants (if any).

6.5 CURRICULUM

1. Cross reference to curriculum or student progress instrument.
2. The evaluation team will conduct a visual inspection.  Provide a copy of the tool inventory / location.
3. Highlight pertinent information in the Advisory Committee minutes.
4. Provide syllabus (with information highlighted), course descriptions, job sheets, student materials, etc.
5.  Provide samples of work order forms, parts order form, and show how time spent on task is recorded.

6.6 STUDENT PROGRESS

1. Provide the school policy on student evaluation, sample of student progress chart, and use an actual record with student identifying information blocked out.

6.7 PERFORMANCE STANDARDS

1. Provide a task sheet or progress chart.
2. Provide the evaluation criteria from the syllabus, progress chart, or task sheet.
3. Provide a task sheet or student progress chart.

6.8 SAFETY STANDARDS

1. and B. Show an example of the safety test, course of study, course outline, posters, etc.
3. Provide the course of study and sample of the safety test.
4. The evaluation team will conduct a visual inspection of markings on guards and lanes, posting of safety rules and signage, and present an example of a student contract.

6.9 PERSONAL STANDARDS

1. The evaluation team will conduct a visual inspection.  Provide instructional materials, class / lab / shop rules.

6.10 WORK HABITS/ETHICS

1. - B. The evaluation team will conduct a visual inspection.  Describe attendance policy, etc.

6.11 PROVISIONS FOR INDIVIDUAL DIFFERENCES

1. Provide ADA information, equipment modifications, differential instruction, and provide an example of an Individual Education Plan (IEP).

6.12 RELATED INSTRUCTION

1. Show syllabus with objectives and examples of tasks where related instruction is provided (OHM’s Law, Pascal’s Law, gear ratio, etc.); SkillsUSA Personal Development Plan, if appropriate.
2. Show copy of instructor teaching credential.

6.13 TESTING

1. Show samples of written tests.
2. Show sample job sheets.
3. Show sample of the rating scale used.
4. Show posters, ASE test registration books, describe provisions made for taking ASE tests.

6.14 EVALUATION OF INSTRUCTION

1. - E.  Provide an explanation of the overall program evaluation policy and plan. Show samples of the instructor evaluation instrument, etc.
6. Highlight pertinent information in Advisory Committee minutes.

6.15 ON-VEHICLE SERVICE AND REPAIR WORK

1. Show task sheets and repair orders. The evaluation team will conduct a visual inspection. 
2. Show course of study and a copy of the student progress chart.
3. Provide a copy of the program policy.
4. Show a sample work order. The evaluation team will conduct a visual inspection. 

6.16 ARTICULATION

1. Show copy of the articulation agreement. Note: this may be N/A.

STANDARD 7 – EQUIPMENT

EQUIPMENT AND TOOLS USED IN THE AUTOMOBILE TECHNICIAN TRAINING PROGRAM MUST BE OF THE TYPE AND QUALITY FOUND IN THE REPAIR INDUSTRY AND MUST ALSO BE THE TYPE NEEDED TO PROVIDE TRAINING TO MEET THE PROGRAM GOALS AND OBJECTIVES.

7.1 SAFETY

1. - B. The evaluation team will conduct a visual inspection. 

7.2 QUANTITY AND QUALITY

1. The evaluation team will conduct a visual inspection. 
2. The evaluation team will conduct a visual inspection of class size and inventory.
3. The evaluation team will conduct a visual inspection. 

7.3 CONSUMABLE SUPPLIES

1. The evaluation team will conduct a visual inspection.  Provide inventory sheets and describe replenishment procedure.

7.4 MAINTENANCE

1. Provide a copy of the preventive maintenance schedule or spreadsheet.

7.5 REPLACEMENT

1. Describe the annual review process and provide an example from the annual survey data and Advisory Committee minutes with pertinent information highlighted.

7.6 INVENTORY

1. Provide the inventory list and describe the control system.

7.7 PARTS PURCHASING

1. If purchase parts, provide a written procedure or parts request form.
2. ETL may discuss this issue with instructor.

7.8 HAND TOOLS

1. Provide an inventory. The evaluation team will conduct a visual inspection. 
2. Explain policy and provide information available for students detailing recommended tool list and vendor visits.

STANDARD 8 – FACILITIES

THE PHYSICAL FACILITIES MUST BE ADEQUATE TO PERMIT ACHIEVEMENT OF THE PROGRAM GOALS AND PERFORMANCE OBJECTIVES.

8.1 TRAINING STATIONS

1. The evaluation team will conduct a visual inspection.  Provide information on class size for each course.

8.2 SAFETY

1. The evaluation team will conduct a visual inspection of the location of signs.
2. The evaluation team will conduct a visual inspection of fire extinguishers. 
3. The evaluation team will conduct a visual inspection and location of posted policy/procedures.
4. The evaluation team will conduct a visual inspection of lighting. 
5. Note inspection schedule, show check list, and highlight pertinent comments in Advisory Committee minutes.
6. The evaluation team will conduct a visual inspection. 
7. The evaluation team will look for the identified vehicle traffic lanes. 

8.3 MAINTENANCE

1. Provide copy of written policy and procedures.

8.4 HOUSEKEEPING

1. - B. The evaluation team will conduct a visual inspection. 

8.5 OFFICE SPACE

1. The evaluation team will conduct a visual inspection.

8.6 INSTRUCTIONAL AREA

1. The evaluation team will conduct a visual inspection. 

8.7 STORAGE

A. - E.  The evaluation team will conduct a visual inspection. 

8.8 SUPPORT FACILITIES

1. - B.  The evaluation team will conduct a visual inspection.

8.9 VENTILATION

1. The evaluation team will conduct a visual inspection and verify the function of exhaust fume removal system.
2. The ETL and team members will interview instructors and students.

8.10 FIRST AID

1. - C. Provide copy of the written policy. The evaluation team will conduct a visual inspection. 

8.11 FACILITY EVALUATION

1. Highlight pertinent information in Advisory Committee minutes.

STANDARD 9 – INSTRUCTIONAL STAFF

9.1 TECHNICAL COMPETENCY

1. A. - D.   Provide information on each instructor, diplomas earned, and copy of ASE Certification.

9.2 INSTRUCTIONAL COMPETENCY / CERTIFICATION

1. Provide a copy of the teaching certificate for each instructor.

9.3 TECHNICAL UPDATING

1. Provide a copy of the inventory of trade publications, service bulletins, etc. The evaluation team will conduct a visual inspection. 
2. Provide certificate, transcript, or completion forms for each instructor.

9.4 FIRST AID

1. Provide a copy of the written policy on First Aid.

9.5 SUBSTITUTE

1. - C. Provide written policy on substitute teachers and schedule for orientation of new substitutes.

STANDARD 10 – COOPERATIVE AGREEMENTS

WRITTEN POLICIES AND PROCEDURES SHOULD BE USED FOR COOPERATIVE AND APPRENTICESHIP TRAINING PROGRAMS.

10.1 STANDARDS

1. Show overall co-op or apprenticeship plan, sample training plan, and the ETL will talk with instructor. This may be N/A.
2. Show a sample agreement. This may be N/A.
3. Show written policy on supervision, identify the person responsible for supervision; the ETL should interview the person who supervises co-op or apprenticeship. This may be N/A.