怎么 and 什么 in Chinese are interrogative substitutes.
They work as placeholders. Their position in the sentence depends entirely on what other phrase they are replacing, in order to compose a interrogative sentence.
In 你怎么知道我的名字 the substitute 怎么 replaces a complement of the verb 知道, which would occur before 知道. You can see this if we rewrite the sentence to be affirmative:
我就是知道你的名字 I just know your name (affirmative with same verb, 就是 can answer the "how" question, and occurs before the verb)
Replying to 怎么知道 while maintaining a simple complement + 知道 to illustrate how the substitute works is kinda hard. A less contrived example might be:
In 你叫什么名字, the substitute 什么 acts a modifier of 名字. It would work also without 名字, so for the purpose of this argument, let's assume you ask 你叫什么. Then you rewrite the sentence to be affirmative as:
Again, the position of the phrase that answers the question is the same as the interrogative substitute.
Yes, there are some exceptions, due to how you can construct question and answers with complex complements, e.g. subordinate clauses, etc. but the rule of thumb is, the interrogative substitute goes in the sentence where its answer would go.