"يخرب بيتك" = "تربت يداك" بالعربية الفصحى

تَرِبَتْ يَدَاكْ أي لصقتا بالتّراب وهي كناية عن الدعاء على شخص بالفقر والخسران {مِسْكِينًا ذَا مَتْرَبَةٍ} ولكن لا يراد بها حقيقتها، فالعرب تقولها لإظهار المحبة والدعاء بالخير.

This classical Arabic expression is often said to express fondness, its contextual meaning is close to "God bless you". Using an expression in a way that contradicts its semantic meaning is common in many languages, like how we say "fat chance" in English to mean "slim chance". Context is always crucial to figuring out the intended meaning.