The paper discusses a novel approach of accelerating the numerical Path Integration method, used for generating a stationary joint response probability density function of a dynamic system subjected to a random excitation, by the GPU computing. The paper proposes the parallelization of nested loops technique and demonstrates the advantages of GPU computing. Two, three and four dimensional in space problems are investigated as a part of the pilot project and the achieved maximum accelerations are reported. Three degree-of-freedom system (6D) is approached by the Path Integration technique for the first time. The application of the proposed GPU methodology for problems of stochastic dynamics and reliability are discussed.