I like to group books by color families (but I make sure to not be too crazy about it because I like it to feel natural and not overly planned). In the photo above you can see that I did a black and white cubby and a turquoise cubby.
If you need more filler in your bookcase, don't be afraid to add magazines. My favorite magazines to decorate with are House Beautiful, Southern Living and Coastal Living. These are all magazines I read regularly, and the ones with the prettiest covers and spines get used in shelves and on cocktail and side tables.
I also like to use various bookends and throw in some photos/artwork. I currently only have one framed photo in the bookshelf, but I intend to add more once we move and I start from scratch. One neat trend that I really like is hanging things on the front of your bookshelf. I hung (with an adhesive 3M hook) a framed print that Tom and I bought in Hawaii on our honeymoon, and it makes me smile every time I see it.
A bookshelf is also a great place to display random collections. In the photo directly above you can see (barely) a shark tooth and a rock that I found on the beach. I didn't want these to just end up in a drawer, so on the bookshelf they go!
If you're using your bookshelf as a room divider, like I am, don't forget the backside! I tried to decorate the back where I could, even though the front is the side that is seen more often. On this side you can see that I added some hour glasses, a vase of flowers, and a lucite "LOVE" sign.
I also hung a mirrored star on the back because I wanted to use it and had no extra wall space for it. 3M picture hanging hooks are the best!