The integration of organic solvent nanofiltration in processes of fine chemistry involving homogeneous metal catalysts has a great potentiel because this eco-friendly process, efficient at molecular scale, does not desactivate the catalyst contrary to conventionnal distillation. The aim of this study is to integrate organic solvent nanofiltration in the process of 10-undecenitrile hydroformylation in toluene. A one-step nanofiltration does not permit to fulfill the goals of the separation, which are to extract the product and to recycle the catalytic system (Rh, biphephos). Simulations of membrane cascades based on experimental data of flux and retention according to the concentration (acquired in the first part of the thesis) highlighted that a four stages cascades with recycling is realistic and competitive for an industrial plant.