The aim of this study was to assess the effects of a series of different surface coated quantum dots (QDs) (organic, carboxylated [COOH] and amino [NH2] polytethylene glycol [PEG]) on J774.A1 macrophage cell viability and to further determine which part of the QDs cause such toxicity. Cytotoxic examination (MTT assay and LDH release) showed organic QDs to induce significant cytotoxicity up to 48 h, even at a low particle concentration (20 nM), whilst both COOH and NH2 (PEG) QDs caused reduced cell viability and cell membrane permeability after 24 and 48 h exposure at 80 nM. Subsequent analysis of the elements that constitute the QD core, core/shell and (organic QD) surface coating showed that the surface coating drives QD toxicity. Elemental analysis (ICP-AES) after 48 h, however, also observed a release of Cd from organic QDs. In conclusion, both the specific surface coating and core material can have a significant impact on QD toxicity.