C: 我不羡慕你。
A: 为什么要羡慕我?
The answer seems improper but rather quite common in daily conversations. It is a technique of using reverse questioning (反問) to - 1) terminate the conservation, either he was under a bad mood or too busy, the tune of speaking won't be too friendly (沒好氣), and was hinting "shut-up, leave me alone". 2) extend the conservation if you have a more than just co-worker relationship with him, and the response can be considered a 俏皮話 in order to receive further talk-backs (反擊).
Note that "我不羡慕你。" has the implication of "幸災樂禍" - "Glad to see you are in a situation I've no envy of, ha ha!" So don't say it to coworkers with who you don't have a close/working relationship with.
In such circumstance, "又加班吶? 真不幸!" should be acceptable to all.