Osteoporosis leads to bone fragility and represents a major health problem in our aging societies. Bone is a quasi-brittle hierarchical composite that exhibits damage with distinct crack morphologies in compression and tension when overloaded. A recent study reported the complex damage response of bovine compact bone under four different cyclic overloading experiments combining compression and tension. The aim of the present work is to propose a mechanistic model by which cracking bone accumulates residual strain and reduces elastic modulus in distinct compressive and tensile overloading modes. A simple rheological unit of bone with two types of cracks is formulated in the framework of continuum damage mechanics. A statistics of these rheological units is then assembled in parallel to compute the response of a macroscopic bone sample in which compressive and tensile cracks are opened, closed or propagated towards failure. The resulting constitutive model reproduces the key macroscopic features of bone tissue damage and delivers an excellent agreement with the four cyclic overloading experiments. The remarkable predictions of the model support the presence of (1) friction between the crack surfaces producing hystereses, (2) an incomplete closure of cracks leading to residual strains, (3) a bridging mechanism of collagen fibrils which failure reduces elastic modulus, and (4) two distinct classes of cracks where compressive cracks have a strong influence on tensile damage and tensile cracks have a limited impact on compressive damage. This work is expected to help improve our understanding of the bone damage mechanisms contributing to skeletal fragility and to foster a proper generalization of this damage behavior in 3D for computational analysis of bone and bone-implant systems.