Search engines don't just look at your site's backinks. They also analyze the "quality" of your site and site quality is also very important to your customers and potential partners like affiliates.