I don't know about you, but around 2:00pm, my brain turns to mush. It stays that way for a few hours. My good times to work are first thing in the morning and late at night. For awhile, my partner and I were blocking off mornings on our calendars so nobody could schedule meetings for us then. We did all our concepting before lunch, then spent the afternoon handling meetings, emails, expense reports, all that low-brain work.
You should figure out what works best for you. If you work best at night when it's quiet, try to get in the habit of sitting down and at least jotting down some ideas then. And if you feel like you're in the mood, don't let it pass you by.
A few months ago, Greg posted the idea from Stephen Covey that our tasks can be divided up by their urgency and their importance. He basically argued that we spend most of our time doing urgent but unimportant tasks, leaving important things with no deadline (all those big ideas that you'll get to one day) on the back burner permanently.
My good buddy Brian Button just sent me this great post from Mark McGuinness that argues essentially the same thing. To him, we get our priorities backwards, doing reactive work (emails, returning calls, etc.) first and leaving our big creative tasks for the dusty hours of the day, when our brains are running on fumes. Thus, our novels never get finished, our websites remain half-vacant and our great side projects never become more than cryptic sticky notes above our desks. I encourage you to read Mark's post. Creative people are fueled by the big ideas. We do ourselves a disservice when we don't follow through with them.
