There has been something of an uproar lately about the FAA changing questions in the knowledge test bank for a couple of their tests without telling anybody. Taking a written test.

It seems that this has resulted in an increase in the failure rate for those tests.  AOPA and NAFI, among others, are complaining to the FAA that the changes are unfair. Hmm.

The tests with the changes in the question bank are for the Fundamentals of Instruction (FOI), Airline Transport Pilot (ATP) and Flight Engineer (FE) ratings.  The AOPA website has an article about the changes with the following quote:

AOPA is not opposed to changes in the knowledge test bank; however, those changes must be coordinated with those providing training for applicants, said AOPA and the National Association of Flight Instructors (NAFI) in a March 3 letter to the FAA. “Unannounced changes in evaluation standards accomplish nothing for learning; it only results in increased student failures, lost time, travel expense and an extra $140 – $150 paid by the students to retake the exam,” said the letter from Kristine Hartzell, AOPA manager of regulatory affairs and Jason Blair, NAFI’s executive director.

OK, I can see that it is AOPA’s job to try and protect pilots from unnecessary costs, etc. But they have also been complaining about the fatal accident rate and the poor condition of the flight instruction industry.

Question: Why would changing the questions in the test bank cause more test failures if the students were being taught the material to the proper level of understanding?

If we are taught to the Correlation Level (refer to the FOI manual if that is an unfamiliar term), a change in the wording of the questions or the questions themselves should have no effect – we know the material well enough to answer correctly. Do you think that maybe we’re just learning what we have to learn in order to pass the test? I admit to studying the material and then getting the test bank and studying only the correct answer. We’ve probably all been there. But is that what we really should be doing?

The FOI information is pretty dull for a pilot – not much ‘real’ aviation in there. But that’s not the purpose of the test – that test is required in order to become a Flight Instructor. It is probably the only information/instruction that we will ever get on how to be a teacher. It sounds like we’re not learning the material very well – and the quality of flight instruction complaints in the recent surveys seem to bear this out.

Isn’t the ATP rating supposed to be the PhD of aviation ratings? Aren’t the pilots who hold that rating supposed to be the most knowledgeable and experienced of all of us? Shouldn’t someone testing for that rating be able to correlate the information well enough to correctly answer at least 70% of the questions on the test?

The FAA publishes the subject matter that they feel is required for the various ratings (14 CFR Part 61). They are limited to testing only those subjects. They publish a Learning Statement Reference Guide that contains the question codes that we receive on our test results report if we miss a question. Each code has its knowledge item for the question – a specific piece of information that we are supposed to know.

We know the subject areas that are going to be tested. If we learn the material well enough, then we shouldn’t need to know the question that is going to be asked.

In any other course that you have taken, did the school or instructor give you the test questions before you took the test so that you could study them? Not in my experience. Yeah, somebody always seemed to have a file of old tests that the instructor had used – but historical results do not guarantee future success.

Bottom line: Learn the material like your life depends upon it – it does.


Added 3/16/2011:
Today Scott Spangler over at Jetwine published a post about this subject – the FAA changing the test questions. He did considerably more research into the details of the FAA actions than I did. I understand his position, but I still don’t see how it changes the need to know the subject matter not just the test questions.