There has been something of an uproar lately about the FAA changing questions in the knowledge test bank for a couple of their tests without telling anybody. 
It seems that this has resulted in an increase in the failure rate for those tests. AOPA and NAFI, among others, are complaining to the FAA that the changes are unfair. Hmm.
The tests with the changes in the question bank are for the Fundamentals of Instruction (FOI), Airline Transport Pilot (ATP) and Flight Engineer (FE) ratings. The AOPA website has an article about the changes with the following quote:
AOPA is not opposed to changes in the knowledge test bank; however, those changes must be coordinated with those providing training for applicants, said AOPA and the National Association of Flight Instructors (NAFI) in a March 3 letter to the FAA. “Unannounced changes in evaluation standards accomplish nothing for learning; it only results in increased student failures, lost time, travel expense and an extra $140 – $150 paid by the students to retake the exam,” said the letter from Kristine Hartzell, AOPA manager of regulatory affairs and Jason Blair, NAFI’s executive director.
OK, I can see that it is AOPA’s job to try and protect pilots from unnecessary costs, etc. But they have also been complaining about the fatal accident rate and the poor condition of the flight instruction industry.
Question: Why would changing the questions in the test bank cause more test failures if the students were being taught the material to the proper level of understanding?
If we are taught to the Correlation Level (refer to the FOI manual if that is an unfamiliar term), a change in the wording of the questions or the questions themselves should have no effect – we know the material well enough to answer correctly. Do you think that maybe we’re just learning what we have to learn in order to pass the test? I admit to studying the material and then getting the test bank and studying only the correct answer. We’ve probably all been there. But is that what we really should be doing?
The FOI information is pretty dull for a pilot – not much ‘real’ aviation in there. But that’s not the purpose of the test – that test is required in order to become a Flight Instructor. It is probably the only information/instruction that we will ever get on how to be a teacher. It sounds like we’re not learning the material very well – and the quality of flight instruction complaints in the recent surveys seem to bear this out.
Isn’t the ATP rating supposed to be the PhD of aviation ratings? Aren’t the pilots who hold that rating supposed to be the most knowledgeable and experienced of all of us? Shouldn’t someone testing for that rating be able to correlate the information well enough to correctly answer at least 70% of the questions on the test?
The FAA publishes the subject matter that they feel is required for the various ratings (14 CFR Part 61). They are limited to testing only those subjects. They publish a Learning Statement Reference Guide that contains the question codes that we receive on our test results report if we miss a question. Each code has its knowledge item for the question – a specific piece of information that we are supposed to know.
We know the subject areas that are going to be tested. If we learn the material well enough, then we shouldn’t need to know the question that is going to be asked.
In any other course that you have taken, did the school or instructor give you the test questions before you took the test so that you could study them? Not in my experience. Yeah, somebody always seemed to have a file of old tests that the instructor had used – but historical results do not guarantee future success.
Bottom line: Learn the material like your life depends upon it – it does.
Added 3/16/2011:
Today Scott Spangler over at Jetwine published a post about this subject – the FAA changing the test questions. He did considerably more research into the details of the FAA actions than I did. I understand his position, but I still don’t see how it changes the need to know the subject matter not just the test questions.
Comments
7 responses to “Another Problem With Flight Instruction”
I could not agree more firmly! AOPA and the instructor’s group should focus their effort on increasing the level of pilot knowledge, meaning mastery of the material. Teach toward knowledge and competency, not test passing. Most CFIs are competent pilots, but not all CFIs are gifted teachers. Passing the test does not assure a safe pilot. With a thorough command of the material, the prospect pilot should be able to process and correctly answer questions – WITHOUT the candidate or the instructor ever having seen them! Although passing the test is necessary, students should focus on mastery. As you so properly point out, Your Life Depends on it! Great post!! Thank you. -Craig
As the holder of a Canadian CPL and someone who started but didn’t finish a Canadian flight instructor rating, it’s always blown me away that the FAA publishes their question banks at all. Transport Canada will start publishing their question banks shortly after Hell hosts an NHL game…
Teaching to the test is NOT the best way to learn or to teach. Having the full question bank published has always seemed like a great way to encourage outright teaching to the test.
Yes, my point exactly – and pretty much explains the failures by just changing the questions while covering the same material.
I would have to agree too that students should not study to the test. Teaching instructors how to teach is very important. It’s always seemed strange to me that the FAA gives out questions.
However, the AOPA is completely correct on the point that the FAA should have stated it’s intentions about changing the test. It is not the fault of these current instructor candidates that almost all of their mentors somewhat blow off in depth study of this kind of of material (like the FOI material).
Further, these tests only test wrote learning. I would think if, when studying test questions, you were sure to review WHY you made a wrong answer, you would get some correlation level of learning. I think these tests could be a little bit better. They could be better at testing that students know the material fairly well with a minimum of the trick questions.
Additionally, the instructor’s handbook, while somewhat improved, is kind of a bit on the terrible side. Maybe that’s the nature of the beast because it’s rather abstract. It needs to be 50% practical examples and less wrote memorization of theories that the average instructor will not gain deep expertise in.
Finally, the primary reason that instruction is so bad is because many are type “A” pilots that don’t really care about sharing a good time with their students (we all know what the “A” stands for). Also, many instructors tend towards the younger side, have no customer service skills, and are not being selected for innate teaching talent.
I am glad I heard they changed the test. At the very least, it WILL motivate me to read, take notes, and pay closer attention to the 2008 version of the handbook.
Thanks guys!
Thanks, you make some very valid points.
Tracy
You had all better learn human resourses, CRM and Safety Management Systems (SMS) also as they are on the new tests.
Carol
All part of being a more professional instructor and a better pilot.
tr