Successful organisms, from an evolutionary standpoint, are those that survive until they can reproduce. To survive the organism must be fit enough to find food, protect itself from predators, and, in the case of sexually reproducing species, attract a mate. Unfit individuals are less likely to survive: either they starve, or, equivalently from an evolutionary standpoint, they fail to mate and pass their genes down to the next generation.
In a large population, harmful mutations are not likely to be reproduced. Other individuals will have a large choice of mates and are not likely to choose a relatively unhealthy mate. In small populations, however, a phenomenon known as ``breeding pressure'' increases the odds that a harmful mutation is reproduced. Fit individuals will have relatively fewer choices for mates, and thus unfit organisms may reproduce.