For both beauty and flavor, consider planting berry bushes

This undated photo shows clove currant flowers in New Paltz, N.Y. Attractive flowers are just one of the assets of clove currant; the flowers are also extremely fragrant and go on to yield tasty, dark currants. (Lee Reich via AP)

No need to choose between planting either an ornamental or a fruiting bush in your yard. Plant both with a single plant! Many berry bushes are pretty enough to be grown as landscape plants.