In this paper we investigate the non-Markovian dynamics of a qubit by comparing two generalized master equations with memory. In the case of a thermal bath, we derive the solution of the recently proposed post-Markovian master equation [ A. Shabani and D. A. Lidar Phys. Rev. A 71 020101 (2005)] and we study the dynamics for an exponentially decaying memory kernel. We compare the solution of the post-Markovian master equation with the solution of the typical memory kernel master equation. Our results lead to a new physical interpretation of the reservoir correlation function and bring to light the limits of usability of master equations with memory for the system under consideration.