Based on the specialty of separation layer, this paper proposes a new contact algorithm with the adoption of massive parallel computation for the study of separation of overburden layers in coal mine production. We have done the real 3D mechanic simulation and get the potential location, shape and volume for separation of overburden layers. Meanwhile, the study takes the area of No. 2 coal mine of Jining as an example, which proves the applicability and validity of this new algorithm.