Baen is the first place everyone mentions; they've been publishing DRM-free ebooks for over 10 years, and have a large free library of ebooks, including a lot of first-in-series works. Baen's ebooks are $6 each (some older ones are as low as $4) and offered in multiple formats.

Fictionwise has a large selection of multiformat ebooks; those are all without DRM. Backlist Ebooks is a collection of authors who have reclaimed their rights & are releasing their own backlists; it skews heavy toward romance and mystery. Links to several more non-DRM ebook stores; while the romance/erotica genres are pretty much king of the ebook world (queen of the ebook world?), there's also several science fiction publishers and a few nonfic publishers.

Some of Amazon's ebooks are sold without DRM; the way to check on the book's listing is the device limit--if it's "unlimited," there's no DRM.