Question :- "I don't really understand why this is a problem"
It is not a "problem" in the sense that it would get you into serious trouble with the law, or cause a riot or make you lose half your Chinese friends. I further submit that it is not a "problem" for you personally. Please read further for my explanation.
Whether in English, Chinese or indeed any language, a non-native speaker is more likely to be excused or sympathetically tolerated for not writing or speaking well in that language. And in most languages too there is the more formal written aspect and the rather informal / colloquial spoken aspect. I understand that foreign students from non-English speaking countries studying in Australian universities do get some allowance for not writing in the Queen's English in their exams. (I was told this by a friend who studied there; so it may not be official policy)
Anyway, coming back to the question, if a non-native speaker were to speak in a formal way in an informal setting, whether in 书面语 Chinese or the language of Shakespeare in English, to a native speaker, the latter would just take it that this foreigner / non-native speaker is in the process of trying to improve his / her proficiency in that language and so would give a "sympathetic allowance" and leave it at that.
But, on the other hand, if a native speaker were to do the same, (i.e. speak using the formal classical structure and words of either 书面语 or Shakespearean English to another native speaker in an informal setting), it is not inconceivable that the native listener would get the impression that the speaker is just showing off, or bordering on being non-sociable.
I suppose that was why someone told you about this 书面语 thing, and I would guess that this person is a native Chinese speaker who sees it as a native Chinese speaker would and not from the point of view of a non-native Chinese speaker who is more "free" and who therefore need not be overly self-conscious or concerned with some amorphous social etiquette, like your goodself.