The sentence refers to a past action at a specific point in time, which requires the past simple tense.
In negative sentences using the past simple, the auxiliary verb "did" is used, followed by the base form of the main verb.
The correct sentence should be:
"He didn't know the answer to the question."
Thus, the correct answer is (B) "didn't know."