Skip to main content

Full text of "DTIC ADA319227: Evaluation of an Out-of-the-Window Air Traffic Control Tower Simulation for Controller Training."

See other formats


Evaluation of an Out-of-the-Window 
Air Traffic Control Tower Simulation 
for Controller Training 


DOT/FAA/AR-96/107 
DOT-VNTSC-FAA-96-14 

Office of Aviation Research 
Washington, DC 20591 



Research and Special Programs Administration 
Voipe National Transportation Systems Center 
Cambridge, MA 02142-1093 


Approved loi guciis reieoast 

L_.... IJalsa aBNi 


Eric Nadler 

Final Report 
September 1996 

This document is available to the public 
through the National Technical Information 
Service, Springfield, VA 22161 




U.S, Department of Transportation 

Federal Aviation Administration 


19961230 041 




NOTICE 


This document is disseminated under the sponsorship of the 
Department of Transportation in the interest of information 
exchange. The United States Government assumes no 
liability for its contents or use thereof. 


NOTICE 

The United States Government does not endorse products or 
manufacturers. Trade or manufacturers’ names appear herein 
solely because they are considered essential to the objective 
of this report. 





REPORT DOCUMENTATION PAGE 


Form Approved 
0MB No. 0704-0188 


Public reporting burden for this collection of information is estimated to average.1 hour per response, including the 
time for reviewing instructions, searching existing_data sources, gathering and maintaining the data needed, and 
completing and reviewing the collection or information. Send comments regarding this burden estimate or any other 
aspect of this collection of information, including suggestions for reducing this burdjsn, to Washington Headquarters 
Services, Directorate for Information Operations and Reports, 1215 Jefferson Dayis Highway, Suite 1204, Arlington. VA 


1. AGENCY USE ONLY (Leave blank) 


2. REPORT DATE 

September 1996 


3. REPORT TYPE AND DATES COVERED 
Final Report 

December 1993 - August 1996 


4. TITLE AND SUBTITLE 

Evaluation of an Out-of-the-Window Air Traffic Control Tower 
Simulation for Controller Training 


5. FUNDING NUMBERS 


FA6L1/A6112 


6. AUTHORCS) 

Eric Nadler 


7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Department of Transportation 

John A. Volpe National Transportation Systems Center 
Research and Special Programs Administration 
Cambridge, MA 02142 


9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 
U.S. Department of Transportation 
Federal Aviation Administration 
Research and Development Service 
800 Independence Avenue, SW 
Washington, DC 20591 


8. PERFORMING ORGANIZATION 
REPORT NUMBER 

DOT-VNTSC-FAA-96 -14 


10. SPONSORING/MONITORING 
AGENCY REPORT NUMBER 

DOT/FAA/AR-96/107 


11. SUPPLEMENTARY NOTES 


12a. DISTRIBUTION/AVAILABILITY STATEMENT 


12b. DISTRIBUTION CODE 


This document is available to the public through the National 
Technical Information Service, Springfield, VA 22161 


13. ABSTRACT (Maximum 200 words) 

This study gathered evidence concerning the potential usefulness of out-of-the-window air traffic control tower 
simulation for training tower controllers. Data were collected from all ten developmental controllers who completed 
simulation training at Chicago O'Hare International Airport during 1994. The simulation included one controller 
position, outbound ground control. An out-of-the-window view was projected on three visual displays which approximated 
the size of actual tower windows. Aircraft were representative of O'Hare, and appeared to move in three dimensions on 
the displays- The simulation could display the entire airport, but only 135 degrees could be seen at a time and no 
inbound aircraft were simulated. After five weeks of simulation training, the trainees became certified on outbound 
ground control in 25% fewer days than trainees who received the same amount of traditional training. However, the 
trainees using the simulation needed only slightly (5%) fewer total hours to become certified on this tower position. 
Evidence suggested that the simulation increased the trainees' working speed, enabling them to work under busier 
conditions, and hence more hours per day. Expert ratings of eight ground control skills based on actual tower 
observations were all higher following simulation training than following traditional training. 


14. SUBJECT TERMS 

air traffic control, simulation, training, air traffic control towers, air traffic control 
tower simulation, outbound ground control, developmental controllers 


15. NUMBER OF PAGES 
60 


16. PRICE CODE 


17. SECURITY CLASSIFICATION 
OF REPORT 
Unclassified 


18. SECURITY CLASSIFICATION 
OF THIS PAGE 
Unclassified 


19. SECURITY CLASSIFICATION 
OF ABSTRACT 
Unclassified 


20. LIMITATION OF ABSTRACT 


NSN 7540-01-280-5500 


Standard Form 298 (Rev. 2-89) 
Prescribed by ANSI Std. 239-18 
298-102 















PREFACE 


This investigation evaluated the effectiveness of an out-of-the-window tower simulation* by 
studying the first year of training conducted in the simulated tower. The author wishes to express 
gratitude to the ten student controllers whose training was studied in this evaluation. Their many 
observations provided an invaluable source of insight about the simulation. 

Jon Bremseth, an O’Hare training specialist, operated the simulation. He previously developed a 
training laboratory using videotapes correlated to flight progress strips, setting the stage for the 
current, interactive simulation training facility. Jon served as the facility contact person for this 
evaluation. He also assisted this evaluation by ensuring that one session was videotaped for each 
trainee each week and by providing the corresponding flightstrips which were used in data 
reduction. He contributed useful observations on the productive use of out-of-the-window tower 
simulation, many of which appear in this report. 

Many other individuals at O’Hare deserve recognition for their outstanding contributions to this 
evaluation. Ellen Jaeger is the Assistant Manager for Training at O’Hare. She and Roy Hillen, 
an O’Hare training specialist, collected preliminary data on trainee errors during their simulation 
training and provided information about the trainees’ prior experience. Tower supervisors Bob 
Kamick and Tim Fitzgerald, and area manager Kevin Markwell provided ratings of trainee skills. 
Matt Dunne, Acting Assistant Manager of Traffic Management provided typical taxi delays, taxi 
times and aircraft rates during an outbound push. The author also recognizes and appreciates the 
critical support provided by Tower Air Traffic Manager, Bill Halleck. 

The author thanks three individuals for their help with technical aspects of this project. Mitch 
Grossberg of FAA/ACT-500 contributed valuable suggestions on the format used to collect 
supervisor ratings. Patricia Pilanen, a tower training specialist at Logan Airport, answered many 
of the author’s questions about tower training and drew the author’s attention to the need for 
ground control trainees to learn to manage flight progress strips without reducing their attention 
to the airport movement areas. Karl Hergenrother of RSPA/Volpe Center created the data 
reduction programs used to reduce duration data from videotape, developed videotape data 
reduction procedures, and contributed valuable observations on the simulation process. 

Objective data reduced from videotape played a key role in this evaluation. This exacting work 
was accomplished by Tufts University engineering psychology students. Ana Pons, Kathleen 
Kim, and Trudi Leone, while they were employed at RSPA/Volpe Center. Bill Voss at ATZ-200 
provided the requirement for this study. Larry Cole at AAR-200 sponsored the effort. Their 
continuing interest and support were critical to the completion of this project. 


1. Cover photograph of TowerPro™ courtesy of Wesson International, Inc, 


iii 



METRIC/ENGLISH CONVERSION FACTORS 


ENGLISH TO METRIC 


METRIC TO ENGLISH 


LENGTH (APPROXIMATE) 

1 inch (in) = 2.5 centimeters (cm) 
1 foot (ft) = 30 centimeters (cm) 
1 yard (yd) = 0.9 meter (m) 

1 mile (mi) = 1.6 kilometers (km) 


AREA (APPROXIMATE) 

1 square inch (sq in, in2) = 6.5 square centimeters (cm^) 
1 square foot (sq ft, ft^) = 0.09 square meter (m2) 

1 square yard (sq yd, yd2) = 0.8 square meter (m2) 

1 square mile (sq mi, mi2) = 2,6 square kilometers (km2) 

1 acre = 0.4 hectare (ha) = 4,000 square meters (m2) 


MASS - WEIGHT (approximate) 

1 ounce (oz) = 28 grams (gm) 

1 pound (lb) = .45 kilogram (kg) 

1 short ton = 2,000 pounds (lb) = 0.9 tonne (t) 


LENGTH (APPROXIMATE) 

1 millimeter (mm) = 0.04 inch (in) 

1 centimeter (cm) = 0.4 Inch (in) 

1 meter (m) = 3.3 feet (ft) 

1 meter (m) = 1.1 yards (yd) 
1 kilometer (km) = 0.6 mile (mi) 


AREA (APPROXIMATE) 

1 square centimeter (cm2) - o.16 square inch (sq in, in2) 

1 square meter (m2) = 1.2 square yards (sq yd, yd2) 
1 square kilometer (km2) = 0.4 square mile (sq mi, mi2) 
10,000 square meters (m2) = 1 hectare (ha) = 2.5 acres 


MASS - WEIGHT (approximate) 

1 gram (gm) = 0.036 ounce (oz) 

1 kilogram (kg) = 2.2 pounds (lb) 

1 tonne (t) = 1,000 kilograms (kg) = 1.1 short tons 


VOLUME (APPROXIMATE) 

1 teaspoon (tsp) = 5 milliliters (ml) 

1 tablespoon (tbsp) = 15 milliliters (ml) 

1 fluid ounce (fl oz) = 30 milliliters (ml) 

1 cup (c) = 0.24 liter (I) 

1 pint (pt) = 0.47 liter (I) 

1 quart (qt) = 0.96 liter (I) 

1 gallon (gal) = 3.8 liters (I) 

1 cubic foot (cu ft, ft3) = 0.03 cubic meter (m^) 
1 cubic yard (cu yd, yd^) = 0.76 cubic meter (m^) 


TEMPERATURE (exact) 

°C=5/9(°F - 32) 


^ VOLUME (APPROXIMATE) 

1 milliliter (ml) = 0.03 fluid ounce (fl oz) 


1 liter (I) 
1 liter (I) 
1 liter (I) 
1 cubic meter (m^) 
1 cubic meter (m^) 


2.1 pints (pt) 

1.06 quarts (qt) 

0.26 gallon (gal) 

36 cubic feet (cu ft, ft^) 

1.3 cubic yards (cu yd, yd^) 


TEMPERATURE (exact) 

°F=9/5(°C) + 32 


QUICK INCH-CENTIMETER LENGTH CONVERSION 


INCHES 0 


CENTIMETERS 0 1 2 3 4 5 6 7 8 9 10 11 12 13 


QUICK FAHRENHEIT-CELSIUS TEMPERATURE CONVERSION 

°F -40° -22° -4° 14° 32° 50° 68° 86° 104° 122° 140° 158° 176° 194° 212° 

III I I 11_1_ J_ 1 _L_I_I_I- 1 — 


— I -1 I r I 1 III I I I I I I 

°C -40° -30° -20° -10° 0° 10° 20° 30° 40° 50° 60° 70° 80° 90° 100° 


For more exact and or other conversion factors, see NIST Miscellaneous Publication 286, Units of Weights and 
Measures. Price $2.50. SD Catalog No. Cl 3 10286. 


updated 8/1/96 


IV 













TABLE OF CONTENTS 


Section Page 


1. INTRODUCTION.1 

1.1 Purpose.1 

1.2 Background.. 1 

1.3 Constraints on the Evaluation.2 

1.3.1 Lack of a Concurrent Control Group...3 

1.3.2 Ongoing Training.3 

1.3.3 Sample Size.3 

2. THE SIMULATION.5 

2.1 Description.5 

2.2 Limitations of the Simulation...6 

2.2.1 Partial Field of View.6 

2.2.2 No Inbound Aircraft.8 

2.2.3 Automated Controller Speech Recognition/Synthetic Pilot Speech.8 

2.3 Precursor to Simulation Training at O’Hare.9 

3. OUT-OF-THE-WINDOW SIMULATION TRAINING METHODS.11 

3.1 Trainees and Schedule.11 

3.2 Simulation Training Procedure.11 

3.3 Modification of Training Methods.13 

4. EVALUATION METHOD. 15 

4.1 Measures of Effectiveness.15 

4.2 Videotaped Training Sessions.15 

5. RESULTS AND DISCUSSION.17 

5.1 Post-Simulation Training Measures.17 

5.1.1 Days-to-Certification.17 

5.1.2 Hours-to-Certification.18 

5.1.3 Tower Supervisor and Area Manager Ratings of Ground Control 

Skills.19 

5.2 Simulation Training Measures.21 

5.2.1 Preliminary Data Considerations and Analysis.21 

5.2.2 Taxi Delay.23 

5.2.3 Taxi Time.24 

5.2.4 Window and Stripboard Scan.26 

5.2.5 Analysis of Instructor Assistance.26 


V 



































TABLE OF CONTENTS (continued) 

Section Eigc 


6. FINDINGS AND RECOMMENDATIONS. 29 

6.1 Training Using the Simulator was Effective.29 

6.2 Training Using the Simulator was Faster and More Effective then Traditional 

Training.30 

6.3 Training Using Tower Simulation is Likely to Show Increased Benefits Upon 

Upgrading.31 

6.4 Current Cost of Recommended Upgrading.34 

6.5 New Applications of Tower Simulation Technology.34 

APPENDIX A; FOCUS GROUP SUMMARY.39 

APPENDIX B: BASELINE DEVELOPMENT OF GROUND CONTROL SKILLS.43 

APPENDIX C: OUTLINE OF RECOMMENDED REQUIREMENTS FOR A TOWER 
SYSTEM ASSESSMENT SIMULATOR.45 


vi 











LIST OF FIGURES 


Figure 


1. SIMULATION EQUIPMENT LAYOUT.5 

2. PARTIAL FIELD OF VIEW.7 

3. AUTOMATED SPEECH RECOGNITION/SYNTHETIC SPEECH: EFFECT ON TAXI 

DELAY.22 

4. AVERAGE TAXI DELAY.23 

5. TAXI DELAY TIMES LESS THAN OR EQUAL TO 30 SECONDS.24 

6. AVERAGE TAXI TIME.25 

7. SCAN TOWARD WINDOW AND STRIPBOARD.26 

8. ASSISTANCE REQUIRED (MAJOR CATEGORIES).28 

9. INSTRUCTOR-CORRECTED AND SELF-CORRECTED COMMUNICATIONS.28 

LIST OF TABLES 

Table 


1. AIRPORT CONFIGURATIONS AND DEPARTURE RUNWAYS AT O’HARE 

AIRPORT.11 

2. TRAINING OBJECTIVES.12 

3. DAYS-TO-CERTIFICATION.17 

4. HOURS-TO-CERTinCATION.18 

5. STRENGTH OF GROUND CONTROL SKILLS FOLLOWING ONE MONTH OF 

SIMULATION TRAINING COMPARED TO BASELINE .20 

6. EQUIVALENT MONTHS OF OJT: SIMULATION TRAINING COMPARED TO 

BASELINE.21 

7. INSTRUCTOR ASSISTANCE BY WEEK OF TRAINING (MEAN NUMBER OF 

ASSISTS).27 


vii 



















viii 




EXECUTIVE SUMMARY 


The objective of this evaluation was to gather evidence bearing on the potential usefulness of 
out-of-the-window ATC tower simulation for training tower controllers. The FAA Research and 
Development Service (ARD) sponsored the development of an out-of-the-window simulation as 
a proof-of-concept demonstration. It was about to be used for the first time to train 
developmental controllers at O’Hare International Airport in Chicago. This simulation was a 
prototype (beta) version of TowerPro''"'^ (Aviation Simulations, Inc.). It had been developed 
specifically to train outbound ground control at O’Hare. The data for this evaluation were 
collected on the progress and relative post-training proficiency of all ten developmental 
controllers who completed simulation training at O’Hare during 1994, the first year of its use. 

The simulation implemented at O’Hare included one controller position, outbound ground 
control. An out-of-the-window view was projected onto three visual displays which 
approximated the size of actual tower windows. Computer-generated aircraft appeared to move 
in three dimensions on these displays at rates which varied with the type of aircraft. The aircraft 
were representative of the types and companies at O’Hare. A session included 30 to 90 aircraft 
in 50 minutes. 

The simulation was limited to three screens, so that only a 135 degree section of the airport could 
be displayed at any one time. Controllers at O’Hare must observe aircraft movements on all 
sides of the tower. In the simulation, instead of turning to look through a different window, the 
trainees used a hand-held control to rotate or “pan” the displayed section to show the relevant 
part of the airport. The simulation was also limited to outbound aircraft: No inbound aircraft 
were included in the simulation scenarios. 

The evaluation was constrained by the lack of a concurrent control group consisting of 
developmental controllers who would have received only traditional training. Instead, 
comparisons were made between the trainees who used the simulation and those who had 
previously been trained at O’Hare, prior to the installation of the simulation. 

The primary measures of simulation effectiveness concerned how long (days and total hours) it 
took a developmental controller to become certified on the outbound ground control position. 

The primary measures were supplemented by ratings made by tower supervisors on specific 
ground control skills, following simulation training. These expert ratings were compared to the 
supervisors’ and an area manager’s recollections of the baseline strength of developmental 
controller skills after varying amounts of time. 

Additional data were reduced from videotapes of the simulation training sessions. The measures 
obtained included taxi delay, defined as the time it took the ground controller to provide taxi 
instructions to an aircraft that was ready to taxi; taxi time, which began when an aircraft received 
taxi instructions and ended when it was instructed to switch radio frequencies to a tower 
frequency; and visual scanning directed toward the “window” or toward the stripboard. 

Instructor comments made during the videotaped sessions were classified according to topic and 
tallied to provide additional indications of the development of specific ground control skills. 


IX 



Also, the first group of trainees’ perceptions of the simulation training were solicited in a focus 
group session. 

The data collected support the following conclusions: 

The out-of-the-window tower simulation used at O’Hare: 

• is an effective tool for training many ground control skills and knowledge, and appears 
particularly effective in increasing trainees’ working speed. 

• is more effective for training outbound ground control than more traditional training 
techniques. 

The effectiveness of the simulation was indicated by the following: 

I. Trainees’ ground control skills increased noticeably during simulation training. 

Taxi delay decreased consistently from the second week to the end of training. 

Stripboard scanning began to consistently decrease after the second week of training, while 
window scanning began to increase after the third week of training. 

The amount of assistance needed for a trainee to properly handle a scenario decreased 
consistently throughout training. 

II. Training using the simulator was more effective than traditional training. 

The developmental controllers who were trained using simulation became certified on the 
outbound ground control position in 25 % fewer days than developmental controllers who were 
trained without simulation. However, using simulation, the trainees needed about the same (only 
5% fewer) hours to become certified on this tower position. 

Expert ratings of eight ground control skills based on actual tower observations were all higher 
following simulation training than following traditional training. 

The difference between the days-to-certification and total hours-to-certification results can be 
explained in terms of working speed: 

The Working Speed Hypothesis: Simulation training increased developmental 
controllers’ working speed, which enabled them to work in the actual tower under a 
wider range of conditions (i.e., under heavier or more complex traffic) and hence for 
more hours per day than with traditional training. 


X 



Recommendations 

Upgrading the simulation could be of value in providing training for the following: 

1. Inbound ground control and coordination between inbound and outbound ground control 
positions 

2. Reflexive, correct communications 

3. Scanning and situational awareness 

4. Understanding what a pilot can see from the cockpit 

5. Smooth transition between window scan and BRITE/ASDE displays 

6. Teamwork 

Simulation training at O’Hare requires use of the simulation facility for roughly six months per 
year. The most productive use of tower simulation at O’Hare would incorporate enhanced 
capabilities and new areas of application that would permit more extensive use. The following 
enhancements and new application areas are discussed: 

1. Individual performance enhancement for current controllers 

2. Team performance enhancement for current controllers 

3. Training in the handling of unusual situations 

4. Optimizing training through the application of experimental training conditions 

5. Tower controller candidate screening 

6. Assessment of new tower equipment, procedures, and airport configurations. 


XI 




xii 




1. INTRODUCTION 


1.1 PURPOSE 

The purpose of this effort was to provide human factors testing and evaluation of an air traffic 
control tower (ATCT) out-of-the-window (OTW) controller training simulation located at 
O’Hare International Airport in Chicago. It was intended to analyze and quantitatively measure, 
to the extent possible, the potential usefulness of devices of this type if they were to be installed 
at selected high density or complex ATC facilities. Goals included: 

• An evaluation of the impact of tower simulation on time-to-certification on position 

• Quantitative estimates of any savings to the user (airlines, aircraft operators) 

• Recommendations for the most productive application of OTW simulators 

• Recommendations for functional specifications of future simulators based on current systems 

• Evaluation of the potential of tower simulation for rapid prototyping to assess the impact of 
new tower equipment, procedures, or configurations. 

1.2 BACKGROUND 

The U.S. air traffic system requires well-trained tower controllers. Controllers receive 
training at the FAA Academy, but the training for high activity (operational level 5) towers 
normally follows years of operational experience as a full performance level controller at less 
active towers, plus many months of training at the high activity tower. Until certified on all 
tower positions, trainees are termed “developmental controllers.” Developmental controllers do 
not work independently at a tower position until they have met the criteria for certification on 
that position. At the nations’ high activity towers, the tower positions typically include flight 
data, clearance delivery, ground control, local control, and supervisor. 

This report concerns outbound ground control training at O’Hare Tower. At O’Hare, controllers 
handle the largest number of aircraft operations of any tower in the country. In 1992, O’Hare 
recorded nearly thirty million enplanements, with a seventy to eighty percent increase in 
enplanements expected by 2010 . The volume and complexity of its traffic make outbound 
ground control at O’Hare arguably the most challenging position to work at and the most difficult 
position to learn of any position at any air traffic facility in the U.S. 

During daily periods of high activity, ground control responsibilities are divided between two 
positions at O’Hare: the inbound ground control position is responsible for arrivals taxiing in to 
their gates; outbound ground control handles departures taxiing out toward a runway. Weather 
accounts for much more delay than any other cause, but according to the FAA Aviation Capacity 

a 

Enhancement Plan, taxi-out consistently accounts for more delay than the other phases of flight . 
This statistic suggests the importance of providing safe and efficient air traffic services to 
outbound aircraft. The same source indicates that O’Hare ranked second in delays of the 55 
airports at which the FAA collects air traffic delay statistics. 


2. FAA. (Sept. 1994). Terminal Area Forecasts FY 1993-2010. FAA-APO-94-11. 

3. FAA. (1994). Aviation Capacity Enhancement Plan. DOT/FAA/ASC-94-1. 



The outbound ground control position at O’Hare requires considerable knowledge and skill. The 
outbound ground controller must first decide which aircraft to call, whether to provide taxi 
instructions or to make a traffic call. Determining the priority of duty requires almost continuous 
attention to the situation on the taxiways, particularly during an “outbound push” (i.e., when 
relatively many aircraft are departing). At the same time, he or she must maintain an awareness 
of the location of aircraft that are ready to taxi. This controller decides which one of O’Hare’s 
eight departure runways to send each departing aircraft to, and in what sequence, decisions that 
must take into account the type of aircraft and its initial route of flight. Then the outbound 
ground controller must decide which series of taxiways the aircraft should use to reach its 
runway. After transmitting the departure runway and taxi route to the pilot, ground control must 
make certain that the pilot understood the instructions, and follows the taxi route. Often the 
outbound ground controller needs to coordinate with the inbound ground controller and local 
controllers when the aircraft is expected to enter their areas of responsibility. 

The outbound ground controller has typically required more than a year of traditional on-the-job 
training (OJT) before a supervisor certifies his or her ability to work independently at this 
position. During this time, an instructor must supervise the developmental controller (trainee) 
while the trainee controls aircraft. At times, the situation on the airport surface is too busy for 
the trainee to handle, requiring the instructor to take over. At other times, there is too little traffic 
or complexity for the trainee to increase the level of his or her skills. The ideal training situation 
is one with enough traffic and complexity to allow the trainee to gain confidence in handling 
difficult situations, but also one that is not so difficult that safety or the expeditious flow of 
aircraft could be reduced. The ideal training situation does not occur often in actuality, so OJT 
proceeds more slowly than if the ideal training situation were always available. 

In an effort to improve the speed and quality of training, the training specialists at O’Hare were 
provided a prototype tower controller training simulator, TowerPro™ (Aviation Simulations, 
Inc.). This prototype was developed for a proof-of-concept demonstration of its potential for 
training outbound ground control at O’Hare. The simulation operator, who was also a training 
specialist, prepared simulation scenarios for training on what he believed would provide the ideal 
amount of traffic and complexity for each trainee. Simulation thus permitted the training of 
ground controllers to always proceed under planned conditions. To the extent that this training 
was sufficiently realistic to transfer to performance in the tower, simulation training was 
expected to reduce training time, particularly for the outbound ground control position. 

1.3 CONSTRAINTS ON THE EVALUATION 

This evaluation was conducted as a field study. As such, it was limited in the use of statistical 
control groups and independent variables. This section describes these constraints and their 
impact on the evaluation. 

• Lack of a concurrent control group 

• Ongoing training 

• Sample size 


2 



1.3.1 Lack of a Concurrent Control Group 


A constraint on the evaluation was the lack of a concurrent control group consisting of 
developmental controllers who would have received only traditional training. No concurrent 
control group was included in the investigation because it was not considered ethical to withhold 
what was expected to be better training from a control group. A possible alternative involved 
comparisons of simulation training at O’Hare to traditional training at another tower. However, 
the results of such comparisons would be suspect due to confounding with other differences 
between the two towers. Instead, comparisons were made between the trainees who used the 
simulation and those who had previously been trained at O’Hare, prior to the installation of the 
simulation. 

1.3.2 Ongoing Training 

The data gathered for this study were collected during the first year the training staff at O’Hare 
used an OTW tower simulator to train tower controllers. Three groups of controllers received 
training over the course of this one-year study. The O’Hare training staff gave eonsiderable 
thought to the way they used the simulator and modified their training practices in an attempt to 
improve the training of each successive group. As a result, the training methods used for the 
three groups differed. This led to instances where it appeared inadvisable to combine the training 
data obtained from the different groups. 

Another constraint on the evaluation resulted from the increasingly difficult demands of the 
training program. Simply stated, more skills were required at the end of training than at the start 
(see section 3.2). Due to this inereasing demand, the trainees needed to increase their skills for 
their performance to remain at the same level. This constraint complicated the interpretation of 
data obtained from simulation training sessions. 

1.3.3 Sample Size 

This evaluation was based on the performance of all ten developmental controllers who received 
simulation training during 1994 at O’Hare Tower. The small size of this sample precluded 
conducting statistical tests on many of the differences observed. 


3 




4 



2. THE SIMULATION 


2.1 DESCRIPTION 

The training simulation evaluated was TOWER/Pro^M, a product of Aviation Simulations, Inc. 

It cost approximately $500,000. It was set up in an O’Hare Tower training room as shown in 
Figure 1. There, it was used to train performance on the outbound ground control position. The 
three visual displays showed aircraft moving about the simulated O’Hare Airport. The displays 
were approximately the size of actual O’Hare Tower cab windows, although they were 
partitioned somewhat differently. Because it presented the airport in a manner virtually identical 
to the airport as seen through the actual tower windows (but see the field of view limitation 
described in section 2.2.1), the device is considered an “out-of-the-window” tower simulator. 
The simulation showed the entire airport including all taxiways and runways, permitting practice 
with all runway configurations. The computer-generated aircraft appeared to move in three 
dimensions at rates which depended upon the type of aircraft. Aircraft crossed from display to 
display without interruption, much as planes seen from the tower cab appear to move along 
taxiways from window to window. The simulation included a representative mix of air carriers 
and propeller-driven types of aircraft. The airport was displayed as if seen from slightly above 
the tower cab because the simulation’s airport image was constructed from photographs taken 
from the roof of the cab. 


Window Display 2 



FIGURE 1. SIMULATION EQUIPMENT LAYOUT 


5 




The airport and aircraft images were projected from a ceiling-mounted source that was separated 
by a curtain from the rest of the training room. The ambient lighting in the training room was 
maintained at a dim level while the simulation was running to minimize reductions in color 
contrast on the window displays. Accurate color representation is important because the colored 
logos and markings on aircraft provide cues to an aircarrier’s company, and controllers use them 
to identify aircraft. 

An instructor and the simulation operator provided training in the simulated O’Hare 
environment. Each instructor was assigned to a pair of developmental controllers, and instructed 
them during simulation training and during OJT. While one trainee assumed the role of 
outbound ground controller, the other served as a “ground meterer.” The ground meterer’s role 
in the simulation was to set up the flight strip for each flight as it was about to enter the 
simulation. 

During the simulation, the trainee stood behind a flight progress strip bay facing the three 
screens. He (all trainees during the one-year data collection period were male) wore a standard 
headset for communication with simulation “pilots.” A camcorder used to collect data for this 
study was located on the trainee’s left and was focused on the trainee to record his voice and the 
direction of his gaze. The ground meterer stood to the trainee’s left near a computer display 
which showed the airport, current aircraft positions, and the simulation clock. The instructor 
stood to the right and behind the trainee. The simulation operator, who was also a training 
specialist, sat behind the instructor at a console with aircraft movement controls and a display 
showing the airport and aircraft locations. The trainee could see neither the ground meterer’s nor 
the simulation operator’s display. 

2.2 LIMITATIONS OF THE SIMULATION 

Every simulation embodies limitations. Following are the limitations of the simulation upon 
which this report is based: 

• partial field of view 

• automated controller speech recognition/synthetic pilot speech 

• no inbound aircraft. 

2.2.1 Partial Field of View 

O’Hare Tower overlooks the airport on all sides, permitting controllers to view all of the 
taxiways and runways, which are found on all sides of the tower. Since the ground and local 
controllers must visually locate each aircraft before providing it with movement instructions and 
clearances, their scanning must include every direction. The simulation could display all of the 
airport, but only 135 degrees of arc was visible at a time through the three window displays. At 
O’Hare, the ground controllers often move about the tower cab for a better view of the airport. In 
the simulation, though, the trainees had to select one 135 degree segment to view at a time 
(Figure 2). They used a handheld control similar to a computer mouse to rotate or “pan” the 
view until the desired segment of the airport could be seen. Trainees could vary the panning rate 
from approximately 45 degrees per second to 150 degrees per second. Panning from a fixed 


6 



position thus took the place of moving around the tower. Thus, trainees needed to learn an 
artificial simulator skill (panning) to participate in simulation training. 



FIGURE 2. PARTIAL FIELD OF VIEW 


It was probably more difficult for the trainees to locate, identify, and track aircraft in the 
simulation than in the real tower because they needed to pan the displayed 135 degree segment of 
the airport to the part that they wanted to see. Some evidence of this difficulty appeared in a 
focus group conducted with the first group of trainees (see Appendix A for a summary). When a 
controller scans the airport from the real tower, he or she physically changes his or her view with 
eye movements, head turns, and body movement. The perceptual system adjusts to produce a 
stable view, a phenomenon known as position constancy". In the simulation, when the trainees 
panned the view to bring a target onto the display, display motion was not accompanied by self- 
motion, so relatively fast panning produced an unstable view. The effect of panning was that the 
trainees could not identify aircraft while panning quickly, but instead needed to either pan slowly 
or wait until the display stopped. 


4. Matlin, M.W. (1988). Sensation and Perception (2nd Ed.). Allyn and Bacon. 


7 



2.2.2 No Inbound Aircraft 


Inbound aircraft were not simulated. Training focused on the outbound ground control position 
and thus required a maximum of practice handling outbound aircraft. Limiting the simulation to 
inbound aircraft also maintained the complexity of the traffic at levels appropriate for new 
developmental controllers. 

2.2.3 Automated Controller Speech Recognition/Synthetic Pilot Speech 

The simulation included automated speech recognition and synthetic speech capabilities. Speech 
recognition was intended to accept eorrectly phrased taxi instructions and then respond 
automatically with the corresponding aircraft movements and an automated pilot readback using 
synthetic speech. The automated system was available, but its use was discontinued because it 
hindered training by plaeing several artificial constraints on trainee speeeh. As a result, speech 
recognition was discontinued during the third week of attempted use by the first group of trainees 
(see Appendix A for trainee observations regarding speech recognition). The following 
constraints were observed: 

1. Trainees needed to remember to pause at the end of each instruction, and to not pause in the 
middle of the instruction. This often interfered with a controller’s speech cadence. 

2. It was impractical to train the speech recognition system to recognize the complete lexicon 
used by ground controllers to assist pilots. If the automated system failed to understand an 
instruction, the utterance did not elicit a simulator response. As an example, it did not 
recognize both grouped and ungrouped numbers in aircraft callsigns. Also, unlike real pilots, 
the simulation did not recognize common taxi instructions which identified the relative 
position of aircraft, such as “Join alfa behind the second MD80 on your left.” When a 
recognition failure occurred, the trainee needed to rephrase and re-issue the instruction, which 
slowed and disrupted performance. 

3. The speech recognition system artificially limited the ways a trainee could issue instructions 
to multiple aircraft because each aircraft needed to respond before another could accept an 
instruction. This limitation interfered with training on the effective use of the ground control 
frequency. 

4. Trainees needed to “train” the speech recognition system to recognize their voices, requiring 
several days that were then not available for training. 

5. Synthetie pilot responses (readbacks and aircraft movements) occurred more slowly than in 
actual operations. 


8 



2.3 PRECURSOR TO SIMULATION TRAINING AT O’HARE 


In 1989, the O’Hare training staff began to use a large screen television to display a videotape of 
the airport taken with an 8 mm camera. Flight progress strips were correlated to the videotape. 
This non-interactive training aid was used successfully to train developmental controllers to issue 
taxi instructions, sequence flight progress strips, and make traffic calls. This precursor of 
simulation training was not interactive. The training specialist could pause the videotape to ask 
what the trainee would do next or how he or she would handle a particular situation, but the 
trainee could not alter the flow of traffic. Each group that was trained using this laboratory aid 
completed a post-training evaluation. Many respondents found the video “lab” to be very 
helpful. 


9 



10 



3. OUT-OF-THE-WINDOW SIMULATION TRAINING METHODS 


3.1 TRAINEES AND SCHEDULE 

Ten developmental controllers completed simulation training at O’Hare during the one-year data 
collection period. Seven of them later became certified on outbound ground control. They 
participated in three groups. The first two groups each consisted of four trainees, and the third 
group initially consisted of three, one of whom was required to interrupt simulation training for 
medical reasons. He completed simulation training after the data collection period had ended. 

Of the seven who became certified on outbound ground control, three were in Group 1, three 
were in Group 2, and one was in Group 3. 

Each trainee participated in the simulation as the outbound ground controller for one hour daily, 
four days per week. The trainees worked in pairs. One pair participated for two hours in the 
morning, and the other pair participated in the afternoon. While one pair was in simulation 
training the other received OJT in the tower. At first OJT consisted of work at the clearance 
delivery and flight data positions. As the trainees’ skills advanced, they began OJT at the ground 
and local control positions. 

3.2 SIMULATION TRAINING PROCEDURE 

Prior to each session, the simulation operator prepared an aircraft schedule including the type and 
company of each outbound plane and the time and gate where it would enter the simulation. The 
aircraft were one of two components in each simulation scenario. The second component was 
the Plan, which corresponded to an airport configuration. The Plan indicated the active runways 
that the trainee was to assign to the aircraft, depending upon their type, company, and gate. At 
the start of a session, the simulation operator announced the Plan that was to be applied. See 
Table 1 for a list of the Plans in use at O’Hare during the year data were collected. 


TABLE 1. AIRPORT CONFIGURATIONS AND DEPARTURE RUNWAYS AT 

O’HARE AIRPORT 


Plan (Configuration) 

Departure Runways 

Weird 

32LatT10, 22L 

X 

32L at TIO, 4L, 9L, some 32R, 14L, 32L full length 

B 

22L, 27L 

14’s 

9L, 9R, 27L 

27’s 

32LatT10, 32R 

9’s 

4L, 9R 

Modified X 

4L, 9R, 32R 

14R/9R 

9L, 22L, some 14L 


11 




Also prior to each session, a paper flight strip was printed for each aircraft showing its callsign, 
initial route of flight, gate, and the time it was scheduled to appear at its gate. The flight strips, in 
standard plastic holders, were arrayed on the ground meterer s table in the order that the aircraft 
were scheduled to enter the simulation. After the session began, the ground meterer monitored 
the simulation time on his display. When an aircraft was scheduled to enter the simulation, he 
set its flight strip in the trainee’s (controller’s) strip bay. 

An aircraft was ready to taxi in response to the simulation operator s movement commands after 
it automatically pushed back from its gate and moved toward the taxiways. The controller 
trainee called aircraft and issued taxi instructions through the headset push-to-talk microphone 
while marking and moving the plane’s flight strip. The simulation operator then responded as 
the pilot of the aircraft, reading back instructions and taxiing the aircraft. The trainee continued 
to issue verbal instructions guiding each aircraft along the taxiways until he instructed the pilot to 
switch radio frequencies to a tower frequency. As the trainee worked the simulated outbound 
traffic, the instructor and simulation operator responded to his decisions with suggestions, 
questions, and comments. The sessions lasted approximately 50 minutes and were followed by a 
brief review. 

In the simulation, the simulation operator (a training specialist) and instructor asked questions, 
made suggestions, and commented on the trainee’s decisions. The simulation operator also 
responded as a pilot (actually, as all of the pilots). In the pilot role, he provided verbal readbacks 
and acknowledgments, taxied the aircraft and, following the handoff to a tower frequency, taxied 
the aircraft to the assigned runway and made it depart. The simulation operator paused the 
simulation when extended discussion appeared necessary. The simulation was paused 
infrequently (once or twice in a typical session); pauses lasted less than two minutes. 

As the trainees’ proficiency increased, they were expected to demonstrate more ground control 
skills and to handle more aircraft. Session objectives became more complex with the progressive 
addition of more advanced skills, as shown in Table 2. The rate at which aircraft appeared in the 
scenarios was gradually increased from one aircraft every two minutes (30 total) to one every 
forty-five seconds (80 total). The airport configuration or Plan was changed daily, but Plan 
Weird and Plan B were used more often in training because they were more frequently 
encountered in actual operations. A change of configuration (e.g., Plan 27’s to Plan Weird) was 
practiced during the last week of training. 


TABLE 2. TRAINING OBJECTIVES 


Phase of Training 

Objectives 

Early 

Issue correct taxi routes, master stripboard management 

Middle 

Assign appropriate runway, issue correct taxi instructions, master 
stripboard management 

Late 

Assign appropriate runway, issue correct taxi instructions, master 
stripboard management, make traffic calls, handle configuration changes 


12 




3.3 MODinCATION OF TRAINING METHODS 


As stated earlier, this study was based on data gathered during the first year the training staff at 
O’Hare used an OTW tower simulator to conduct training. They modified some training 
techniques over the course of the year. For example, whereas the first two of the three groups of 
trainees obtained practice on all O’Hare air carriers throughout their training, the training 
methods used for the third group emphasized one half of the airport at a time by first 
concentrating on the American Airlines rush. One anticipated benefit of this training strategy 
was that it would reduce the amount of panning required and thus reduce any distration panning 
might have created. 

A set amount of training was planned for the first two groups. In contrast, the third group was 
given training until a criterion was reached. As a result, one Group 3 developmental controller 
received four weeks of simulation training, while the other received this training for three weeks. 
The criterion consisted of a demonstration that the trainee could: 

- issue correct taxi routes 

- effectively manage the stripboard 

- assign the appropriate runway 

- issue correct taxi instructions 

- make traffic calls when needed. 

The simulation’s automated speech recognition/pilot response system was used only during the 
first three weeks of the first group’s four-week simulation training (see section 2.2.3). The 
second and third groups received simulation training without automated speech 
recognition/synthetic speech. Group 2 received one week more simulation training than Group 1 
because it was not necessary to take away the time needed to “train” the automated speech 
recognition system from simulation training. 


13 



14 



4. EVALUATION METHOD 


4.1 MEASURES OF EFFECTIVENESS 

This report describes an evaluation of the first year of training using an OTW simulator to teach 
the skills required to perform outbound ground control at O’Hare Tower. Since an important 
goal of using the tower simulation in training was to reduce training time, the primary evidence 
used to evaluate its effectiveness was the time required for the simulation-trained 
controllers to become certified on outbound ground control, compared to facility records. 
To augment these data, the perceptions of tower supervisors were used to compare simulation- 
trained to traditionally trained developmental controllers on a variety of ground control skills. 
Their ratings helped to identify the specific ground control skills that simulation training 
benefited and the strength of these skills following simulation training. 

The evaluation was also based on objective evidence reduced from videotapes of the simulation 
training sessions. Three types of evidence were reduced from the videotapes: 

• Taxi Delay: how long the pilot waited before receiving taxi instructions 

• Taxi Time: how long aircraft were under the control of the ground controller 

• Scan: the proportion of time a trainee’s gaze was directed toward the window displays or 
toward the flight progress strip bay. 

Instructor comments found on the videotapes were quantitatively analyzed to provide additional 
indications of the development of specific ground control skills and to gain a better 
understanding of the simulation training process. 

4.2 VIDEOTAPED TRAINING SESSIONS 

Taxi time, taxi delay, scanning, and instructor comment data were reduced from videotapes of 
training sessions. One training session for each developmental controller was videotaped each 
week for later analysis. The videotaped sessions were conducted as normal simulation training 
sessions. Some exceptions were made for the purposes of the analysis: The same Plan (Plan X) 
was used in the videotaped sessions during the first and last weeks of simulation training for 
Group 1. The same Plan (Plan Weird) and the same aircraft rate (1.5 per minute) was used in all 
of the second group’s videotaped sessions. 

The training staff increased the aircraft rate for Group 1 each week during the four week 
simulation training period. The rate increased from .5 aircraft per minute during the first week to 
2 aircraft per minute during the last week. The videotaped problems for Group 2 all included 90 
aircraft and used “Plan Weird,’’ one of the two most common configurations of the eight 
configurations used at O’Hare (the other is Plan X). The roughly 90 aircraft per hour rate 
approximates the departing aircraft rate in an actual outbound push. These parameters were kept 
constant in the recorded sessions for the purpose of this investigation. Training specialists 
determined the parameters used in the regular training sessions, depending upon the trainee’s 
skill level and particular training needs. 


15 




16 



5. RESULTS AND DISCUSSION 


5.1 POST-SIMULATION TRAINING MEASURES 

5.1.1 Davs-to-Certification 

The time (i.e., the number of days and total hours) it took developmental controllers to become 
certified to work independently at the outbound ground control position was the primary measure 
used in this study. Days-to-certification was measured from the date a developmental controller 
began working full time in the tower, having completed simulator training, to the date a tower 
supervisor certified him or her on outbound ground control. The hypothesis was that the 
simulation-trained developmental controllers would take fewer days than non-simulation-trained 
developmental controllers to become certified. The test of this hypothesis involved a comparison 
between simulation-trained controllers and non-simulation-trained controllers on days-to- 
certification on outbound ground control. The non-simulation or “traditionally trained” group 
had received four weeks of training in the videotape lab described in section 2.3. Their days-to- 
certification was also measured from the date they began working full time in the tower. 

Only data from those non-simulation-trained controllers who began training after January 1, 1993 
were included in the no-simulation group. Prior to January, 1993, O’Hare certified controllers on 
the outbound and inbound ground control positions together. Since simulator training was only 
given on outbound ground control, the days taken for certification on that position alone were the 
most relevant. Thus, the seven controllers who started training in or after January, 1993, were 
compared on days-to-certification to the seven controllers who became certified on outbound 
ground control following simulator training®. This comparison is illustrated in Table 3. 


TABLE 3. DAYS-TO-CERTIFICATION 



Days-to-Certification | 


N 

Mean 

Standard Deviation 

Simulation 

7 

148.57 

49.65 

No Simulation 

7 

198.71 

52.52 


Table 3 shows that the simulation-trained developmental controllers took 50.1 days less, a 25.2 
% reduction in the number of days-to-certification on outbound ground control. The difference is 
statistically significant, t (12) = 1.84 (p < .05), in a one-tailed test. This finding supports the 
hypothesis that developmental controllers who received simulation-training take fewer days to 
become certified on the outbound ground control position than non-simulator trained 
developmental controllers. 


5. Three developmental controllers were certified on outbound ground control following the year in which the data 
were collected. Including this data would have produced a mean of 162.6 days and a standard deviation of 49.85 
days. 


17 



5.1.2 Hours-to-Certification 


The second part of time-to-certification is the total number of hours required. The hypothesis 
was that the developmental controllers who received simulation-training would take fewer hours 
to become certified than non-simulator trained developmental controllers. The test of this 
hypothesis involved a comparison between the same two groups of developmental controllers 
that were compared in section 5.1.1. 

Table 4 gives the number of hours simulation-trained developmental controllers required for 
certification on outbound ground control and the hours-to-certification required by trainees 
without simulator training. They are approximately equal (a Mann-Whitney test^ failed to reveal 
a significant difference, p > .05). Therefore, simulation training was effective in increasing 
trainees’ skills to a point where the trainees could acquire the needed amount of OJT in fewer 
days, but it did not appear to reduce the total amount of OJT needed. 

TABLE 4. HOURS-TO-CERTIFICATION 



Hours-to-Certification 


N 

Mean 

Standard Deviation 

Simulation 

7 

84.93 

21.18 

No Simulation 

7 

89.42 

12.53 


It will be recalled from section 1.2 that when the density of traffic requires a faster working speed 
than a developmental controller can produce, the instructor must take over the position. The 
developmental controller then loses an opportunity to increase his or her ground control skills 
through practice. Accordingly, a hypothesis that would explain the time-to-certification results is 
that simulation training enables trainees to work more rapidly and thus to handle higher traffic 
densities in OJT. This can be called the working speed hypothesis. 

The working speed hypothesis suggests that with simulation training, developmental controllers 
can obtain more hours of OJT per day, resulting in a need for fewer days of OJT than without 
simulation training. That simulation training did not appear to affect hours-to-certification 
suggests that simulation training does not address at least some critical ground control skills. 
These skills apparently develop independent of working speed so that an increased working 
speed does not reduce the number of hours of OJT needed to develop these other skills. The 
post-simulation-training supervisor ratings presented in the following section (5.1.3) and the 
training session measures presented in section 5.2.5 provide evidence related to the identification 
of these skills. 


6. This nonparametric test was used because the homogeneity of variance assumption underlying the more powerful 
parametric t-test was not met, as indicated by the standard deviation data presented in Table 4. 


18 




5.1.3 Tower Supervisor and Area Manager Ratings of Ground Control Skills 


The measures of time-to-certification provide overall indices of the benefits that can result from 
simulation training, but these measures do not indicate the effect of simulation training on 
specific skills. Supervisory (expert) rating data were obtained to permit an examination of the 
skills exhibited during initial outbound ground control performance following simulation 
training. The strengths of these skills were compared to a baseline that represents the 
development of ground control skills without simulation training. 

The baseline was constructed from ratings of typical OJT progress for traditionally trained 
average and fast learners on eight categories of ground control skill. An O’Hare Tower area 
manager and two O’Hare Tower supervisors made the baseline ratings. Each of the raters had 
worked as O’Hare instructors, area manager, or supervisors for at least six years. Thus, all had 
evaluated ground control skills at O’Hare prior to the use of simulation training. 

The baseline was constructed from the supervisors’ and area manager’s recollections. This 
procedure was chosen because it was not possible to obtain the baseline ratings from direct 
observation. For each skill category, the supervisors estimated the strength of the skill for a 
developmental controller with one week to 12 months experience in the tower. They used five- 
point scales to make their ratings in each of the eight skill areas. The mean baseline ratings for 
“average learners” and “fast learners” and the meaning of each point on each of the 5-point scales 
appear in Appendix B . 

The supervisors were asked to provide ratings on the same eight skills for the simulation-trained 
developmental controllers following each group’s completion of simulation training. The ratings 
for a trainee were to be made as soon as possible following sufficient observation of his work in 
the tower. In all cases they were completed within three weeks following the end of simulation 
training for the individual, after the supervisor had observed the trainee’s ground control skills 
for at least one hour. The ratings for Group 1 were made two weeks after the supervisors and 
area manager provided the baseline data. 

Table 5 presents the rated strengths of the eight skill categories following one month of 
simulation training compared to the strengths of the same skills following one month of 
traditional training. The table presents the mean ratings for the seven developmental controllers 
who became certified on outbound ground control. Table 6 shows how many months of OJT it 
would take average or fast learners to reach the same level of skill that the simulation-trained 
controllers attained within three weeks following simulation training. Both baseline and 
simulation groups received approximately one month of training prior to OJT. 


19 



TABLE 5. STRENGTH OF GROUND CONTROL SKILLS FOLLOWING ONE 
MONTH OF SIMULATION TRAINING COMPARED TO BASELINE 



Simulation 

Baseline (No Simulation) 

Skill 

Strength (1-5) 

Average Learner: 
Strength (1-5) 

Fast Learner: 
Strength (1-5) 

Visually scans 
airport surface 

3.21 

2.0 

2.7 

Maintains efficient 
traffic flow 

3.21 

2.0 

1.7 

Maintains aircraft 
identity 

3.71 

2.0 

2.7 

Working speed 

3.71 

1.3 

2.7 

Effectively manages 
shipboard 

2.89 

1.7 

2.0 

Missed/delayed 
traffic calls 

3.0 

1.0 

1.7 

Communication is 
clear and concise 

3.36 

1.7 

3.0 

Makes unnecessary 
transmissions 

2.93 

1.0 

2.0 

Overall 

3.25 

1.59 

2.31 


Table 6 shows that according to the area manager and tower supervisors whose impressions were 
obtained, the skills of developmental controllers who received simulation training were advanced 
compared to the skills of those who received traditional training. Early in OJT, simulation- 
trained developmental controllers demonstrated ground control skills considered equivalent to 
traditionally trained average learners following five months of OJT. 

Simulation training was particularly effective in training working speed, where the mean rating of 
3.71 indicated that the trainee could work at a speed that was more than adequate for moderate 
workload (3.0) and close to adequate for high workload (4.0). This finding is consistent with the 
working speed hypothesis proposed in the preceding section. This explanation suggested that 
simulation training increased working speed, enabled the trainee to handle higher traffic 
densities, and hence permitted the trainee to work traffic more hours per day. 


20 









TABLE 6. EQUIVALENT MONTHS OF OJT: SIMULATION TRAINING COMPARED 

TO BASELINE 



Simulation 

Baseline (No Simulation) 

Skill 

Months of OJT 

Average Learner: 
Months of OJT 

Fast Learner: 
Months of OJT 

Visually scans 
airport surface 

<1 

4.42 

1.51 

Maintains efficient 
traffic flow 

<1 

5.02 

3.7 

Maintains aircraft 
identity 

<1 

5.42 

2.07 

Working speed 

<1 

6.67 

4.02 

Effectively manages 
stripboard 

<1 

4.63 

2.38 

Missed/delayed 
traffic calls 

<1 

5.0 

3.0 

Communication is 
clear and concise 

<1 

4.17 

2.17 . 

Makes unnecessary 
transmissions 

<1 

4.9 

2.46 

Overall 

<1 

5.03 

2.66 


5.2 SIMULATION TRAINING MEASURES 

Each developmental controller was videotaped once each week during simulation training. The 
following objective data were then reduced from the videotapes: taxi delay, taxi time, and the 
percentages of time a trainee’s scan was directed toward the “windows” or toward the simulation 
stripboard. 

Taxi delay and taxi time provide measures of general planning ability and efficiency. Training 
was expected to result in reduced taxi delay and taxi time. Stripboard scan time is a measure of 
the attention required to stay organized by marking and moving flight strips. Controllers are 
trained to minimize stripboard scanning because it takes time from scanning the situation on the 
airport movement areas. Thus training was expected to increase window scanning and to 
decrease stripboard scanning. 

5.2.1 Preliminary Data Considerations and Analysis 

The Group 3 training data were excluded from analysis for several reasons. In Group 3, only two 
developmental controllers of the three who began simulation training completed it: one needed 
to postpone further training for medical reasons after three weeks. The use of training-to-a- 
criterion instead of a previously set number of weeks of training (see section 3.2) resulted in 
different amounts of training given to the remaining two trainees who completed simulation 


21 





























training. In view of these and other differences between Group 3 and the other two groups, it 
appeared inappropriate to combine the three groups’ training data. Group 3 provided insufficient 
data (a total of eight sessions for three trainees) to conduct a meaningful separate analysis of 
Group 3 alone. 

The Group 1 data required preliminary analysis to assess any impact of the automated speech 
recognition/synthetic speech system on the measures of training. The automated system was 
used during the first two to three weeks of Group 1 training (see section 2.2.3). It appeared to 
lengthen taxi delays when it was operating due to the need for the trainees to repeat taxi 
instructions and watch to see if indeed the aircraft began to move as instructed. By requiring 
unusually lengthy attention to individual aircraft it also was likely to have interfered with the 
scan measures. 

An effort was made to quantify the effect of using the speech recognition system even though it 
was necessary to compare across trainee groups. Figure 3 shows the mean taxi delay for Group 
1, during the first two weeks of automated system use (its use was discontinued during the third 
week, resulting in its use by some but not all trainees that week). The figure also shows taxi 
delay for the first two weeks of Group 2 and Group 3 training, combined for comparison to 
Group 1. The Week 1 scenario included 30 planes for Group 1, in contrast to 90 planes for 
Group 2 and Group 3, so the apparent advantage for the automated system is most likely an 
artifact. The Week 2 scenarios were more comparable: Group 1 was given 80 planes, nearly as 
many as the 90 planes given to Group 2 and Group 3. Using the Week 2 data, comparison of the 
first group while using the automated system with the second and third groups suggests that the 
automated system added an average of 1.77 minutes to the Group 1 taxi delay, roughly doubling 
this delay. Since the automated system appears to have adversely affected the Group 1 training 
data for the first two or three weeks, the Group 1 data was not analyzed further. 


05:02 

04:19 

03:36 

02:53 

02:10 

01:26 

00:43 

00:00 


FIGURE 3. AUTOMATED SPEECH 
RECOGNmON/SYNTHETIC SPEECH: 
EFFECT ON TAXI DELAY 


□ Automated 
■ Manual 



Weekl Week 2 

Weeks of Training 


22 




Due to the preceding considerations, the only training data analyzed came from the videotaped 
sessions of the second group of trainees. The four trainees in Group 2 each received simulation 
training for five weeks. One session per trainee per week was videotaped. Each session lasted 
approximately 50 minutes. Since the taxi delay, taxi time, and scan measures were all based on 
many individual observations, these data were considered to provide a sufficient basis for 
analysis. 

A quantitative analysis of instructor comments on these 20 tapes supplemented the training data. 
This analysis was conducted to provide additional indications of the development of particular 
ground control skills and a better understanding of the simulation training process. 

5.2.2 Taxi Delay 

The time it takes the ground controller to provide taxi instructions to an aircraft that is ready to 
taxi is its taxi delay. At O’Hare, most aircraft receive taxi instructions in less than 30 seconds, 
which is considered a “good” taxi delay time at this facility^. Taxi delay provides a general 
indication of a controller’s ability, including the ability to identify the gates and/or alleys used by 
air carriers at O’Hare, communicate efficiently, plan efficient taxi routes, and prioritize actions. 

In the simulation, taxi delay was defined as the time interval that began when an aircraft was 
metered (i.e., when the ground meterer placed its flight strip on the strip bay) and ended when the 
aircraft received taxi instructions. It did not apply to an aircraft that was handed off by one 
ground controller to another ground controller (one aircraft in the videotaped scenarios). 

Figure 4 shows the mean Group 2 taxi delay, by week of training. It demonstrates that following 
an initial rise, mean taxi delay decreased monotonically from a high during Week 2 of almost 2.5 
minutes to a low during Week 5 of under one minute. This considerable decrease in mean taxi 
delay was accompanied by a corresponding increase in the percentage of taxi delays provided in 
at most 30 seeonds (Figure 5). 


FIGURE 4. A’VERAGE TAXI DELAY 



7. Matt Dunne. FAA O’Hare Tower. Personal Communication. 


23 












FIGURES. TAXI DELAY TIMES LESS 

THAN OR EQUAL TO 30 SECONDS 





DU A) - 

1 50% 
H 40% • 


► 

X 




^ 30% < 

^ 20% - 
o 

b 10% - 

pL, 

no/. 



^ 




U A) d 

1 

1 2 3 4 J 

) 


Weeks of Training 



5.2.3 Taxi Time 

Taxi time is a rough estimate of the time an aircraft spends taxiing and so it provides an 
indication of fuel cost to the user as well as an indication of how well a controller plans taxi 
routes. Taxi time varies with the distance from the departure gate to the point of handoff (usually 
near the departure runway), and hence with the type and company of the aircraft. Because it also 
varies with the distance to the appropriate, active departure runway, taxi time also varies with 
airport configuration. Following are typical taxi times for O’Hare* when the airport is either in 
its “Plan B” or its “Plan Weird” configuration (note that taxi time is reduced in the simulation 
due to the absence of inbound aircraft): 



Departure Runway 

Taxi Time (Minutes) 

United 

Eastbound 

22L 

8.5 

Westbound 

32L 

3 to 4 

American 

Eastbound 

22L 

6 

Westbound 

32L 

8 

General Aviation 

Eastbound 

22L 

<1 

Westbound 

27L 

9 to 10 


8. Matt Dunne. FAA O’Hare Tower. Personal Communication. 


24 









As measured in the simulation videotapes, an aircraft’s taxi time began when it was given taxi 
instructions and ended when it was handed off to the local controller. The airport configuration 
or Plan was held constant (Plan Weird) in the Group 2 sessions videotaped. 

Figure 6 shows the mean taxi times by week of training for Group 2. It is evident that taxi time 
remained relatively stable during the five weeks of training. The taxi times measured were 
considerably lower than most of those considered typical at O’Hare. It is likely that the times 
were low in part because no inbound aircraft were simulated. It is not clear why taxi time did not 
reflect the benefit of simulation training; however, it is possible that 

- taxi time depends mainly on the distance from the departure gate to the appropriate 
runway, with little effect of controller skill. 

- the ground controller has more flexibility in how aircraft are handed off to the local 
controller, the event which closed taxi time in this study. Depending upon the situation, a 
handoff could occur in the same message as a taxi instruction, or it could wait until a later time 
when the trainee had an opportunity to make several handoffs in a single transmission. Time-to- 
runway (if measured) might show better sensitivity to training effects than time-to-handoff. 

- the longer taxi time durations, compared to the length of taxi delays, made them more 
likely to include the time taken by instructors’ assistance and comments. 

- increases in skill were counteracted by increases in training program demands. Some of 
the later training demands such as traffic calls and splitting fixes could affect taxi time more than 
taxi delay. 



25 






5.2.4 Window and Stripboard Scan 


This pair of measures describes the length of time each trainee’s visual scan or gaze was directed 
toward the simulation’s visual displays (i.e., toward the simulated tower windows), the 
stripboard, or elsewhere in the training room (typically toward an instructor). Window scanning 
and stripboard scanning were of particular interest because of anecdotal reports that some 
trainees spend too much time looking at flight strips and consequently insufficient time looking 
through tower windows at the aircraft movement areas. 

Stripboard management began to require less visual attention after the second week of training. 

It decreased between the second and fifth weeks from 51.4% to 44.8% of the total time. Window 
scanning began to increase after the third week from 44.0% to 50.2% of the total time. The 
direction of these moderate changes in gaze is consistent with the hypothesis that simulation 
training improves stripboard management skills. This evidence is presented in Figure 7. 


FIGURE?. SCAN TOWARD WINDOW 
AND STRIPBOARD 



5.2.5 Analysis of Instructor Assistance 

The constraints under which this study was conducted (see section 1.3) require a research 
strategy in which different types of evidence are examined for consistent trends. Accordingly, 
videotaped instances of instructor assistance were analyzed to provide converging evidence on 
the development of specific ground control skills. 

Instructors made 782 discernible suggestions and comments during the 20 Group 2 training 
sessions that were videotaped and used in this evaluation. This instructor advice was examined 
to determine whether the trainees required less advice as training progressed. Each instructor 
comment was classified according to topic to provide additional indications of the development 
of particular ground control skills. 


26 







The instructor comments were classified into twelve categories. Six were designated as “major 
categories” and six were designated as “minor” on the basis of frequency. The major categories 
included 90.15% of the comments. For each week of training, the mean amount of advice 
(number of comments) in each of the major and minor categories is shown in Table 7. Figure 8 
shows the effect of training on the average amount of assistance (per major category) trainees 
required to perform the problem. A trainee required roughly half as much assistance by the end 
of simulation training. 

Table 7 shows reductions in advice on stripboard management and taxi routes, which appear to 
have reached asymptotes (minimum levels) by the end of simulation training. Priority of duty, 
(i.e., knowing which aircraft to call next) required instmctor comments primarily during the first 
two sessions, after which comments appeared at a much reduced rate. Situational awareness is 
the only major topic that appears to have required a fairly constant level of instructor attention 
throughout training. 


TABLE 7. INSTRUCTOR ASSISTANCE BY WEEK OF TRAINING (MEAN NUMBER 

OF ASSISTS) 


Categories Of Assistance 

Weeks of Training 

Major Categories 

1 

2 

3 


5 

Stripboard Management 

10 




3.25 

Situational Awareness 

5.25 



4.5 

4.75 

Priority of Duty 

10 

9 

3.25 

3 

2 

Communications 

9.75 


7.5 


6 

Taxi Routes 

9.75 

9 


5.5 

5.5 

Traffic Calls 

4.75 

3.5 

3.5 

2.75 

1 


Minor Categories 

1 

2 

3 

4 

5 

Splitting Fixes 

.5 

3 


ira 

1.75 

Timely Taxi Instructions 

.5 

1 

.25 


.75 

Handoffs to Local Controller 

.75 

.25 


0 

0 

Aircraft Identification 




1 

.25 

Sequencing Aircraft 

.75 

.25 

1.25 

.5 

.5 

Runway Assignment 

.5 


.25 

.5 

0 


27 











































As training progressed, instructors made fewer comments on the developmental controllers’ 
communications. The highest number of comments were made during the first and second 
sessions. Compared to an average of approximately ten comments made in those initial sessions, 
the final session shows an approximate 40% reduction. Over the same period, the trainees 
increased the number of communication errors they caught and corrected themselves by more 
than 60%. Figure 9 presents these results. 


FIGURE 9. INSTRUCTOR-CORRECTED 



1 2 3 4 5 

Weeks of Training 


28 








6. FINDINGS AND RECOMMENDATIONS 


This section summarizes and draws conclusions from the evidence presented in the preceding 
section. This evidence was gathered during the first year of training using a prototype out-of-the- 
window tower training simulator at O’Hare International Airport in Chicago. The simulator was 
used to train the skills and knowledge required to perform outbound ground control at O’Hare. 
The evidence supports the following conclusions: 

The out-of-the-window tower simulation used at O’Hare: 

• is an effective tool for training many ground control skills and knowledge; it appears 
especially effective in increasing trainees’ working speed. 

• is more effective for training ground control than more traditional training techniques. 

6.1 TRAINING USING THE SIMULATOR WAS EFFECTIVE 

During training, the ground control performance of the developmental controllers studied 
consistently improved, beginning with the second week of training. Following a month of 
simulation training, their ground control performance (in the actual tower) was rated as better 
than the performance of traditionally trained developmental controllers (see section 6.2). The 
following findings indicate improvements during the simulation training sessions, but do not 
permit comparisons between simulation and traditional training. The findings from comparisons 
between simulation and non-simulation training (listed in section 6.2) suggest that the 
improvements in ground control skills described in this section were due either to simulation 
training alone or to a combination of simulation and concurrent non-simulation training. 

Finding: Taxi delay decreased consistently from the second week to the end of training. 

The evaluation found a 50% decrease in average taxi delay duration, with a tripling of the 
percentage of taxi delays under 30 seconds in a comparison of the second and fifth (last) weeks 
of training. The monotonic downward trend was evident during each intervening week. 

Finding: Stripboard scanning began to consistently decrease after the second week of training, 
while window scanning began to increase after the third week of training. 

Moderate improvements in these two objective measures were found. Stripboard management 
began to require less visual attention after the second week of training, and ultimately decreased 
from 51.4% to 44.8% of the total scan time, a 7.7% decrease. Window scanning increased after 
the third session, from 44.0% to 50.2%, a 6.2% increase. 

Finding: The amount of assistance needed for a trainee to properly handle a scenario decreased 
consistently throughout training. 


29 



The evaluation found a decrease of approximately 50% in the number of instructor assists from 
the first to the last week of training. The number of assists decreased in every major category of 
ground control skill. The major categories included Stripboard Management, Situational 
Awareness, Priority of Duty, Communications, Taxi Routes, and Traffic Calls. Stripboard 
Management and Priority of Duty showed the largest decreases. 

6.2 TRAINING USING THE SIMULATOR WAS FASTER AND MORE EFFECTIVE THAN 
TRADITIONAL TRAINING 

Developmental controllers became certified on the outbound ground control position in fewer 
days following a month of simulation training. Supervisors’ ratings of their ground control skills 
indicated that they were further advanced, compared with the results of traditional training. 

Finding: The developmental controllers who were trained using simulation became certified on 
the outbound ground control position in fewer days, but required about the same total number of 
hours, as developmental controllers who were trained without simulation. 

Using facility records, comparisons were made on the number of days and the number of.hours of 
OJT taken to become certified on the outbound ground control position. The evaluation found 
that it took the simulation-trained developmental controllers an average (mean) of 149 days to 
become certified on the outbound ground control position. In contrast, it took the developmental 
controllers who became certified on outbound ground control prior to simulation training 199 
days, a statistically significant difference of approximately 25%. The two groups involved in 
these comparisons had’ received approximately the same amount of training prior to OJT. 

The difference in days-to-certification found between developmental controllers trained using 
simulation and those trained without simulation did not appear to extend to total hours-to- 
certification. The evaluation found that simulation-trained developmental controllers took only 
5% less time in terms of total hours-to-certification. This pair of results provides part of the 
evidence favoring a working speed hypothesis. 

The Working Speed Hypothesis: _____ 

Simulation training increased developmental controllers’ working speed, which enabled them to 
work in the actual tower under a wider range of conditions (i.e., under heavier or more complex 
traffic) and hence for more hours per day than with traditional training. Thus, simulation 
training decreased the number of days but not the total number of hours needed for certification 
on position. ______ 


Additional evidence in support of the working speed hypothesis is found in the supervisor ratings 
of eight skill categories, where working speed was rated highest compared to a baseline of 
traditional training. It received a mean rating of 3.71, which indicates that the trainee could work 
at a speed that is more than adequate for moderate workload (3.0) and close to adequate for high 
workload (4.0). Baseline ratings for controllers given traditional training were 1.3 for average 
learners and 2.7 for fast learners. The working speed hypothesis is also consistent with the 
finding of a 50% drop in taxi delay during the course of simulation training. 


30 




That simulation training did not appear to affect hours-to-certification suggests that simulation 
training did not address at least some critical ground control skills (see the following section 6.3). 
These skills apparently develop independent of working speed so that an increased working 
speed does not reduce the number of hours of OJT needed to develop these other skills. 

Finding: Supervisor ratings made after the completion of simulation training were higher than 
the ratings of traditionally trained developmental controllers. 

In general, the ratings made by tower supervisors indicated that within three weeks following 
simulation training, developmental controllers exhibited ground control skills in the actual tower 
comparable to a traditionally trained “average learner” after five months of OJT or to a “fast 
learner” after 2.7 months of OJT. The supervisors rated all eight categories of ground control 
skill higher following simulation training than following traditional training, even when 
compared to fast learners. Simulation training appeared most effective in increasing working 
speed. 

6.3 TRAINING USING TOWER SIMULATION IS LIKELY TO SHOW INCREASED 
BENEFITS UPON UPGRADING 

One difficulty of traditional OJT for outbound ground controllers is that, at times, the volume of 
traffic and its complexity can become so high that the trainee cannot proceed without potentially 
compromising the safe and efficient flow of aircraft to their runways. At these times, the trainee 
must leave his or her post and allow the instructor to take over. This reduces the amount of time 
for OJT on that day. The decreased taxi delays and higher supervisor ratings found in this study 
are consistent with the notion that simulation training increased the working speed of 
developmental controllers to the point where they could benefit from OJT even under conditions 
of moderate-to-high density and complexity. 

The 25% difference in days-to-certification following simulation training was not accompanied 
by a commensurate reduction in hours-to-certification. These results suggest that some crucial 
ground control skills are not learned in simulation training as currently implemented. To the 
extent that these skills can be identified and simulation training enhanced, extended^, or 
upgraded to support teaching them, one can expect additional benefits. Recommendations in this 
section are offered under the assumption that the FAA decides to proceed with tower controller 
training simulation. 


9. Many of the recommended simulation enhancements and new application areas (section 6.4) involve extending 
simulation training. This would increase training staff time and costs. The simulation’s speech recognition and 
synthetic speech capability was intended to enable a trainee to practice without a simulation operator continually 
present. When the simulation speech system was operating, aircraft would automatically follow their assigned 
routes, and pilots would read back instructions and clearances automatically, using synthetic voice. Problems 
encountered in the use of the evaluated simulation’s speech recognition and synthetic speech systems are described 
in section 2.2.3. If these problems are solved, autonomous capabilities could reduce the scheduling and financial 
costs associated with some of these enhancements and applications. Autonomous simulation capabilities could also 
increase the accuracy of assessments made in a simulated tower environment by increasing control over test 
conditions. 


31 




The evidence obtained in this evaluation suggests that upgrades are required to train or to 
increase the effectiveness of training on the following skills: 

1. Inbound ground control and coordination between inbound and outbound ground control 
positions 

2. Reflexive, correct communications 

3. Scanning and situational awareness 

4. Understanding what a pilot can see from the cockpit 

5. Smooth transition between window scan and BRITE/ASDE displays 

6. Teamwork 

Inbound ground control and coordination between inbound and outbound ground control 
positions. 

Without inbound aircraft, the simulation could not provide the complexity of the taxi patterns at 
O’Hare. This limitation reduced the maximum difficulty of the training problems, and it thus 
reduced the number of weeks the simulation could present challenging problems for training. All 

of the upgraded capabilities listed below that involve extending the duration of simulation 
training require a capability for inbound aircraft. 

Ground control training conducted in a simulation lacking inbound aircraft could have led to the 
development of unrealistic^expectations of the pace, decisions, and actions needed to move from 
one point to another because the outbound aircraft did not have to stop at the runways and 
taxiways used for arrivals. Also, without inbound aircraft, trainees did not have an opportunity 
to learn the coordination skills needed to work outbound ground control in tandem with an 
inbound ground controller. 

Recommendations: 

• Upgrade simulation technology to support inbound aircraft 

• Upgrade simulation technology to include an additional position to be used for inbound 
ground control 

• Extend the duration of simulation training. 

Reflexive, correct communications. It appears that extended simulation training could improve 
trainee communication skills. As the emphasis during the month of simulation training changed 
from fundamental to advanced skills, so did the content of controller communications and their 
associated phraseology. This continual change in what was required of trainee communications 
could have produced the finding that more instructor assistance was provided in the 
communications category than in any other category during the last recorded training session. A 
related finding was that the total number of corrections in the communications category (self¬ 
corrections and instructor corrections) remained fairly constant throughout training. A trend 
toward less instructor assistance on communications after the second week of training suggests 
that trainees were learning better communication skills, but at end of simulation training, 
communications skills were among the least advanced, compared to traditional training, in 
supervisors’ ratings. Together, these findings suggest that additional training could further 
promote the reflexive, correct communications that controllers require at airports like O’Hare, 
which are characterized by high aircraft density and complex traffic patterns. 


32 




Recommendation: 

• Extend the duration of simulation training. 

Scanning and situational awareness. Some results indicated that technical and/or procedural 
aspects of simulation training could be enhanced to increase the amount of time developmental 
controllers scan the airport. Objectively, scanning out the simulation windows increased 6.2% 
during the last two weeks of training. During this time, the amount of instructor assistance on 
situational awareness decreased, but it remained almost as high as at the start of simulation 
training. In supervisor ratings made soon after the completion of simulation training, scanning 
the airport was a skill rated less than average in strength, and less advanced compared to the 
other skills rated. These results suggest that scanning and situational awareness began to 
improve relatively late, and could benefit from additional simulation training. Possibly, the need 
to pan the displayed view of the airport may have slowed the integration of scanning into larger 
units of ground control skill, and may have limited the transfer of scanning skills from the 
simulation to the actual tower. 

Recommendations 

• Enhance the simulation to simultaneously display the entire airport. 

• Extend the duration of simulation training. 

Understanding what a pilot can see from the cockpit. Situational awareness for ground and local 
controllers includes an awareness of what pilots are likely to see at any given moment. This 
awareness is important when communicating instructions which commonly refer to aircraft 
relative to the pilot’s position, as in “Join alfa behind the MD80 off your left.” Situations arose 
during simulation training which suggested that ground controllers need training on what pilots 
are likely to see. For example, an instructor asked, “Which company do you think he’s looking 
at?” On another occasion, an instructor commented on a trainee’s instruction: “He may never see 
American off his left.” A simulation capable of showing a realistic pilot’s eye view could 
advance training on this aspect of situational awareness. 

Recommendation 

Provide a “pilot’s eye view” capability that will realistically represent the view of a pilot from the 
cockpits of particular types of aircraft. 

Smooth transition between window scan and BRITEIASDE displays. 

Developmental controllers could use current tower simulation to learn to translate aircraft 
locations as displayed on BRITE and ASDE systems from/to their locations as seen through the 
tower windows. Training in this advanced skill could lead to increased situational awareness. 

Recommendations 

• Upgrade the simulation technology to provide an interface to BRITE and ASDE displays 
showing aircraft coordinated to the simulation. 

• Extend the duration of simulation training. 


33 




Teamwork. The simulation permits training for only one tower position. With enhancements, 
the simulation could be used to train developmental controllers to participate in the coordinated 
team effort required to control aircraft at O’Hare. Teaching teamwork and coordination skills 
requires the simulation to simultaneously show the entire airport. This capability would enable 
two or more controllers to view different parts of the airport at the same time. 

Recommendations 

• Upgrade simulation technology to support inbound aircraft. 

• Upgrade simulation technology to include an additional position for inbound ground control 
or local control. 

• Enhance the simulation to simultaneously display the entire airport. 

• Extend the duration of simulation training. 

6.4 CURRENT COST OF RECOMMENDED UPGRADING 

The recommended upgrades are all currently available from the U.S. manufacturer of the 
prototype, Wesson International (Aviation Simulatons, Inc. is a joint venture of Wesson 
International, Inc. and the British Bruce Artwick Organization, Ltd.). Costs of the personal 
computer-based system depend primarily upon the number of controller or pseudo 
pilot/simulation operator stations, and on the number of projection screen displays. All current 
systems include simulated BRITE and ASDE displays and “pilot’s eye view” capabilities. A 
system that includes all recommended upgrades (eight displays for a 360 degree view and three 
positions - two controller stations and a pseudo pilot/simulation operator station) would currently 
cost $1,050,950. A new system representative of the one evaluated at O’Hare (three displays for 
a 135 degree view, and one controller position, and a pseudo pilot/simulation operator station) 
would currently cost $495,950. 

6.5 NEW APPLICATIONS OF TOWER SIMULATION TECHNOLOGY 

Simulation training requires use of the simulator for roughly six months per year. Out-of-the- 
window ATC tower simulators offer additional potential uses which could be scheduled during 
the rest of the year, when the simulator is not required for training. The first topics concern 
advanced training for current full performance level (FPL) controllers. Sections follow on the 
use of tower simulation for candidate screening, and for the assessment of new tower training 
methods and technology and the evaluation of new automation systems, procedures, and airport 
configurations. 

Individual performance enhancement for current controllers. The simulation evaluated does not 
appear able to provide sufficient volume and complexity to increase the skills of current FPL 
controllers at O’Hare or at other high density or high complexity airports. It would be essential 
to add a large volume of inbound aircraft to increase the difficulty of the training scenarios if 
training is to benefit current FPL controllers. This training could possibly enable them to provide 
better services and safety and to develop strategies for handling more traffic. 


34 





Team performance enhancement for current controllers. The current one-position simulation 
does not provide opportunities to train tower controllers to work more effectively together as a 
team. A multiple-position simulation with sufficient volumes of inbound and outbound aircraft 
would permit practice and training on situations requiring coordination and teamwork. The 
objectives of this training would be similar to those of crew resource management or CRM^®. 

For example, they might include improvements in end-of-shift briefings, workload distribution, 
situational awareness, management of abnormal situations, and error surveillance. 

Training in the handling of unusual situations. Using simulation, it would be possible to train 
tower controllers to recognize and properly handle situations rarely or infrequently seen in actual 
operations. Without operational experience, simulation would appear to offer the best way for 
controllers to learn to contend with the overall consequences of such situations for airport traffic 
control. Such situations include emergency landings requiring passenger evacuation, hijacked 
aircraft or other aviation security events, landings of diplomatic aircraft, aircraft taxiing on 
runways incapable of supporting their weight, fuel spills, birds or other wildlife on runways, 
unusual and dangerous airport weather (e.g., icy runways, windshear emergencies, or tornadoes), 
mechanical failures on aircraft, emergency medical flights, and aircraft with similar callsigns 
approaching a runway. Procedures for many of these situations appear amenable to simulation 
training and practice. 

The optimal conditions for training. Out-of-the-window tower simulation technology offers the 
ability to control the training environment to a greater extent than the operational environment 
permits. The training staff at O’Hare took advantage of this new capability during the first year 
of simulation training by tailoring the volume of aircraft and the amount of guidance presented to 
the individual, and by modifying their training techniques between groups of trainees (see section 
3.3). 

This ability to control the training environment could lead to further efforts toward optimizing 
tower controller training. Training specialists could apply laboratory findings on the effects of 
experimental training conditions. For instance, in one study (Vidulich, Yeh, and Schneider, 
1983^^), a group of subjects that practiced a simulated air traffic control task that ran at twenty 
times real time performed better in some aspects (and the same in all other areas) than a group 
that practiced at real time. Algorithms for time-compressed information presentation have 
recently been developed . The possibility of applying the lessons of such developments clearly 
depends upon the availability of adequate tower simulation facilities. 

Tower controller candidate screening. Training resources are best allocated to the most highly 
qualified candidates. Current selection procedures consider only how a candidate appears on 
paper. The tower personnel who must select among candidates for a limited number of training 


10. Taggart, W.R (1994). Crew resource management: Achieving enhanced flight operations. In N. Johnson, N. 
McDonald, and R. Fuller (eds). Aviation Psychology in Practice, Aldershot, Hants, U.K., Averbury. 

11. Vidulich, M., Yeh, Y., and Schneider, W. (1983). Time-compressed components for air-intercept control skills. 
Proceedings of the 27th Meeting of the Human Factors Society. 

12. Guckenberger, D., Guckenberger, L. Luongo, F., Stanney, K., and Sepulveda, J. (April, 1995). Above-real-time 
training and the hyper-time algorithm. Dr. Dobb’s Journal, 52-61. 


35 



slots have no direct knowledge of how well the candidates can perform under realistic task 
demands. They also lack clear evidence of motivational factors which can lead a candidate to 
expend special effort at preparation, such as studying the layout of the airport and surrounding 
airspace. In a realistic simulation, candidates can demonstrate their current knowledge and skills 
without affecting operations. Ideally, the tower staff would observe a candidate’s demonstration 
of prerequisite knowledge and skills, and the ability to benefit from training in a simulation 
screening process which extends over several days. A draft document which describes the 
organization of a training candidate screening process using simulation is available from O’Hare 
Tower^^. 

Assessment of new tower equipment, procedures, and airport configurations: Cost-benefit 
studies. 

New tower automation systems, new procedures such as coded taxi routes, and new airport 
configurations (often involving new or extended runways) promise to increase airport capacity 
and to improve airport safety and the services provided by tower controllers. Assessment is 
needed to determine the extent to which the goals of each new system, procedure or 
configuration are reached, to determine its effect on the overall tower environment, and to assess 
proposed design modifications. Although the assessment of each new system can include unique 
considerations, controllers’ visual, manual, pilot communications and coordination workload, 
and their response times, errors, decision making, and situational awareness are common 
considerations. Such assessments must first be made in the context of the current tower 
environment to provide a baseline for comparison with the new automation system, airport 
configuration, or procedure. 

Some pertinent measures cannot be made in an operational environment because they could 
interfere with tower operations or because the necessary control over the situation is lacking. 
Examples include testing controller response time to identify the cause of alarms and respond 
appropriately, to identify and correct a controller error, or to identify and correct pilot deviations 
from controller instructions. In these cases, the initiating events occur too rarely in actual 
operations and would have to be simulated. It would also appear impractical to test the actual 
consequences of automated taxi route decisions, or to determine whether controllers can safely 
monitor them. The measurement of situational awareness technically requires stopping the 
situation to collect data*'*. Other measures, such as the frequency of manual activity and data 
input errors, are best measured automatically through direct connections to the device, 
connections which would require fabrication. In general, the most accurate and relevant 
measurements can be made in an operational situation, in the presence of realistic workload, 
distractions, time demands, and stress. However, as these examples suggest, the operational 
situation may not be suitable to the required measures, requiring instead the use of global or 
subjective measures which could lead to less accurate conclusions. 


13. FAA/Great Lakes Region/O’Hare Tower. (Undated Draft). O’Hare Air Traffic Control Tower Performance 
Verification Program. Author. 

14. Endsley, M. (1995). Measurement of situational awareness in dynamic systems. Human Factors, 37(1), 65-84. 


36 



Out-of-the-window tower simulation could provide a more convenient and practical testbed for 
the assessment of new tower automation systems than operating towers. It would appear 
particularly useful in the assessment of automation systems to be used by local and ground 
controllers, whose work depends upon viewing aircraft movement areas. A list of recommended 
requirements for an assessment simulator is presented in Appendix C. 


37 





38 



APPENDIX A: FOCUS GROUP SUMMARY 


This focus group was conducted with four developmental controllers in the first group after they 
had 7 to 10 hours working at the outbound ground control position, following 20 hours (4 weeks) of 
simulation training. Following is an edited summary of the session. 

In what ways is actual ground control different from the simulation? 

- Because of the need to scroll the simulation it is harder to keep track of AC in peripheral vision 
and to know when to transfer the AC. 

- Simulation workload is constant because it presents planes at a regular pace, unlike the real tower 
where the workload occurs in "clumps". There is no "breathing time" in the simulation. 

- The simulation's graphic resolution is sometimes "grainy" and its colors less intense, adding a half 
second to the time it takes to identify aircraft. For example, one must reason from type (737) to 
company (not American, therefore United) to check yourself; upstairs you would know instantly. 

- More small planes. Eagles and Gulfs, in actual traffic than in simulated. Maybe needs updating. 

In what ways was the simulation an effective teaching tool? 

- Learning taxi routes. 

- It eliminates worry about real aircraft moving around and the training specialist stepping in to fix 
something instantly or to make something work. It provides a less pressured, more relaxed 
environment instead of needing to fix it instantly. 

- It permits control over the pace of the problem. Upstairs when its slow its too slow; when busy it 
is too busy; there seems to be no happy medium. 

Was the pace of the simulation optimally challenging throughout all four weeks? 

- It began a little too slow at 1 plane per minute, and too fast at the end (1 per 30 sec) when learning 
to make splits. The speed jumped up sometimes before I caught on. 

If there was another month of simulation what would you like to cover? 

- Splits, and more varied scenarios, including surges and rest, different planes and different rushes, 
instead of always starting up west and ending up with west. 

-Rehearsal of actual rushes and regularly scheduled flights in all configurations. 

Any advice for the next group of developmentals to receive simulation training? 

- Focus on managing stripboard. 

- Don't worry about making mistakes or looking stupid, and ask questions instead of pretending to 
understand. 

How might the simulation be used better? 

- Instructors should use the pause feature more to allow evaluation of what has just occurred, in 
smaller chunks, identifying problem and solution. 

- Speed up the problems at the beginning to make them more realistic. 

- Reduce the speed of a problem when first increasing its complexity. 

- Use 1 or 2 instructors full time instead of many instructors because they teach different techniques 
for doing the same thing. 


39 



According to the trainees, the following topics are learned better with live traffic than through 
simulation training (although with "additional tweaking" the simulation could help with these): 

- Split fixes. 

- Relative taxi speeds. 

- Location: It is easier to see planes waiting for departure on the approach end of a taxiway in the 
real tower, than on the simulator. - Visual scan: One learns to move around in the actual tower, 
whereas one always looks straight ahead in the simulation. One does learn where to scan in the 
simulation. 

- Aircraft identification: The simulation picture is not sharp enough to identify companies. 

Did the simulation help you learn... 

Efficient traffic flow? 

- Yes. It provides practice on maintaining continuous attention and not wasting time 
Making traffic calls? 

- No. [sorry, I did not ask for elaboration] 

Increased working speed? 

-Yes. 

Coordination with inbound ground controller? 

- No, because there was no inbound traffic to coordinate. 

Flight strip marking and stripboard management? 

- It provided good help, although more work is needed to get the instructor, student, and simulation 
working at the same speed. 

Phraseology and correct frequencies? 

- It was helpful for learning to use correct frequencies. The voice recognition was too picky, failing 
to accept an instruction if the student said "the" instead of "a", spoke too loud, soft, or fast. It 
distracted from what the student was trying to learn. 

Dealing with similar callsigns? 

- It never came up. 

Avoiding unnecessary verbiage? 

- The training helped, but not the simulation's voice recognition. 

Any other aspects of your present work with actual traffic where you are finding that the simulation 
helped? 

- The location of gates and where to look for airplanes calling to taxi out. It sped up the learning 
process. 

- "I can't really put it in numbers, but I feel a lot better about approaching the ground control 
learning after a month here.... I feel well ahead of the game here." 

- "I think it was very beneficial...! think it helped in all areas...it greatly helped with stripboard 
management, stripmarking, and phraeology, not from voice recognition, but from practice and 
repetition, saying the routes over and over. 


40 




How did the speed of the simulation compare with actual traffic? 

- The speed of the simulation problems was in same range over slow periods, but it did not get as 
fast as it does upstairs; the 7:30 rush does not compare to the simulation. 

- It is hard to compare without inbounds in the simulation because inbounds can become confused 
with your [outbound] traffic (which adds complexity). "You are looking at more than your guys 
upstairs; you are looking at everything." 

What improvements would be most important? 

- A 360 degree screen would help the most. 

- Simulation operators to run remote positions; one to run inbounds, another to be the pilot for the 
outbounds, and then there would be the instructor, watching. You could set up traps where certain 
pilots miss their instructions. 

- Voice recognition would be valuable if it allowed a student to practice without imposing on 
others' time. This would allow for the additional practice needed to develop habits which in turn 
would translate to time saved. Four weeks were not enough to develop stripmarking and stripboard 
management skill habits; "I didn't get to writing the number on the strip, looking out the window 
and moving it, all without thinking. I still had to think about writing the number on the strip, 
moving the strip, and sequencing it, and figure out which one's an American and which one's a 
United and putting them between those two. The habits didn't come for me as fast as they said they 
should because if you do it only one hour per day for 20 days...." 


41 



42 




APPENDIX B: BASELINE DEVELOPMENT OF GROUND CONTROL SKILLS 


“Average Learner” 


Ground Control Skill 

1 

wk 

1 

mo 

2 

mo 

M 

6 

mo 





Visually scans airport surface 

2.0 

2.0 

2.7 

3.0 




5.0 


1 “Never 2=Seldom 3=Often 4=Almost Always 5=Always 


Maintains efficient traffic flow 

1.3 

2.0 

2.0 

2.7 





1 “Never 2=Seldom 

3=Often 

4“Almost Always 

5 “Always 

Maintains aircraft identity 

1.7 

2.0 

3.0 

3.0 

4.0 

4.7 

4.7 


1 “Never 2=Seldom 

3“Often 

4“Almost Always 

5“Always 

Effectively manages stripboard 

1.3 

1.7 

2.3 

2.7 

3.7 

4.0 

4.7 

5.0 

1 “Never 2“Seldom 

3=Often 

4“Almost Always 

5“Always 

Working speed 

1.3 

1.3 

1 

2.7 

3.3 

4.3 

4.7 

5.0 


l=Very Slow 2=Adequate for Low Workload 3=Adequate for Moderate Workload 
4=Adequate for High Workload 5=Adequate for Very High Workload 


1 Missed/delayed traffic calls 

1.0 

1.0 

1.3 

2.7 

3.3 

4.3 


4.7 

l“Many 2=Some 3“Few 

4“Almost None 

5= 

=None 

Communication is clear & concise 

1.7 

1.7 

2.3 

3.3 

4.0 



Ea 

1 “Never 2“Seldom 3=Often 

4“Almost Always 

5 “Always 

Makes unnecessary transmissions 

1.0 

1.0 

1.7 

2.3 

3.7 

4.0 

4.3 

4.7 


l=Many 2=Some 3=Few 4=Almost None 5=None 


43 






































“Fast Learner” 


Ground Control Skill 

1 

wk 

12 4 6 

mo mo mo mo 

8 10 12 

mo mo mo 





Visually scans airport surface 

2.0 

2.7 3.7 4.0 4.7 

5.0 5.0 5.0 

l=Never 2=Seldom 

3=Often 

4=Almost Always 

5=Always 

Maintains efficient traffic flow 

1.3 

1.7 2.7 3.3 4.3 

5.0 5.0 5.0 

1 =N ever 2=Seldom 

3=Often 

4=Almost Always 

5=Always 

Maintains aircraft identity 

2.3 



1 =N ever 2=Seldom 

3=Often 

4=Almost Always 

5=Always 

Effectively manages stripboard 

2.0 

2.0 3.0 3.7 4.7 

5.0 5.0 5.0 

l=Never 2=Seldom 

3=Often 

4=Almost Always 

5=Always 

Working speed 

2.0 

2.7 2.7 3.7 4.0 

4.7 5.0 5.0 


l=Very Slow 2=Adequate for Low Workload 3=Adequate for Moderate Workload 
4=Adequate for High Workload 5=Adequate for Very High Workload 


1 Missed/delayed traffic calls 

1.3 

1.7 

2.7 

3.3 

4.0 

4.3 

4.7 

4.7 

l=Many 2=Some 3=Few 

4=Almost None 

5= 

=None 

Communication is clear & concise 

2.3 

3.0 





4.7 

5.0 1 

l=Never 2=Seldom 3=Often 

4=Almost Always 

5=Always 

Makes unnecessary transmissions 

2.0 

2.0 

2.7 

3.7 

4.3 

4.7 

4.7 

5.0 1 


l=Many 2=Some 3=Few 4=Almost None 5=None 


44 





















































APPENDIX C: OUTLINE OF RECOMMENDED REQUIREMENTS FOR A TOWER 

SYSTEM ASSESSMENT SIMULATOR 


A. Display 

1. Room space sufficient to replicate largest tower 

2. Moveable realistic-size window displays each showing the view from 

one side of the actual tower 

3. All tower window views displayed simultaneously 

4. Photographic color images of entire airports, readily updatable 

5. Color photographic images of aircraft of types and companies appropriate to 

airport, readily updatable 

6. Inbound and outbound aircraft in local controller airspace at realistic speeds 

which vary with aircraft type 

7. Aircraft taxiing in and out at realistic speeds which vary with aircraft type 

8. Capable of exchanging the photograph of one airport for another airport 

9. Maximum volume of aircraft equal to future enplanements predicted for simulated airport 

10. Aircraft and gates individually scheduled 

11. Ability to remove, modify, and rename current taxiways and runways and to create new or 
extended taxiways and runways 

12. Sessions recorded for playback 

B. Control (by human operator or automated system) 

1. Individual aircraft movements along specified taxiways and runways 

2. Takeoff/landing roll and liftoff/landing at realistic runway positions and speeds 

3. Verbal responses transmitted through controller headsets 

C. Positions (include furniture and devices) 

1. ATC (8): two ground control, two local control, flow manager, flight data, clearance 

delivery, supervisor, and 

2. Simulation operator/pilot positions (4); two respond to local control; two respond to ground 

control, or 

3. Automated speech recognition/synthetic pilot and one simulation operator who can 

override the automated pilot systems 

D. Interface to systems under test 

1. Actual current and new system user interface 

2. Capable of simulating inputs to current or new systems 

3. Capable of simulating faults in new systems 

4. Adaptable to new automation system interfaces 

5. Quickly reconfigurable with different combinations of equipment 

E. Interface to controller performance data logging eq ui pment 

1. Source of input (device used) 

2. Identity of data or command correctly or incorrectly entered 

3. Time-logging of all data collected, synchronized to simulation 


45 




F. Additional data collection 

1. Videotape from each window display location, synchronized to simulation 

2. Audiotape from each control position, synchronized to simulation 

3. Record of simulation operator commands and time entered 


46