'与' in '与我' means 'with'.
It can't be omitted
[用中文][与我联系][不是一个好主意]
[using Chinese] [(to) contact with me] [is not a good idea]
English grammar:
It is not a good idea to contact with me using Chinese
The sentence is grammatically correct. Nothing wrong with that.
You can replace the relative clause [用中文] with other context and the grammar is still correct.
For example:
[在午夜] [与我联系][不是一个好主意]
[At midnight] [ contact with me] [is not a good idea]
English grammar:
It is not a good idea to contact with me at midnight
Note:
联络 is the better choice than 联系 for 'to contact' in this case because 联络 only means 'to contact' while '联系' could also mean 'to connect' as in 'connect and form a relationship'