The ability of cell-penetrating peptides (CPPs) to cross cell membranes has found numerous applications in the delivery of bioactive compounds to the cytosol of living cells. Their internalization mechanisms have been questioned many times, and after 20 years of intense debate, it is now widely accepted that both energy-dependent and energy-independent mechanisms account for their penetration properties. However, the energy-independent mechanisms, named "direct translocation", occurring without the requirement of the cell internalization machinery, remain to be fully rationalized at the molecular level. Using artificial membrane bilayers, recent progress has been made toward the comprehension of the direct translocation event. This review summarizes our current understanding of the translocation process, starting from the adsorption of the CPP on the membrane to the membrane crossing itself. We describe the different key steps occurring before direct translocation, because each of them can promote and/or hamper translocation of the CPP through the membrane. We then dissect the modification to the membranes induced by the presence of the CPPs. Finally, we focus on the latest studies describing the direct translocation mechanisms. These results provide an important framework within which to design new CPPs and to rationalize an eventual selectivity of CPPs in their penetration ability.