For all the talk about Ayers, I haven't heard anyone make an argument as to why people should care other than 'judgment and character'. The Ayers association and Obama's lies regarding it do reflect badly on those things, but they are so abstract that the media and non-conservatives can rationalize them away.
There are three specific reasons the Ayers connection is relevant to voters.
If Obama is friends with Ayers, Wright, Rezko, etc., what does that tell us about who he would appoint to positions in his administration and get advice from?
At the Annenburg Challenge, Obama and Ayers gave money to groups to radicalize children with liberal propaganda. If he used $50 mil to brainwash our children instead of teaching them reading and writing, should we trust him with the federal education budget?
Obama doesn't necessarily share the views of Ayers and Wright, but the fact that he is friends with them demonstrates how far to the left he is. 80% of Americans would be so offended by their views they would never be on friendly terms with them, let alone work with them or consider them mentors.
The fact that he would tolerate their radical anti-American views demonstrates his values are such that he doesn't find them offensive regardless of whether he agrees with them. He must have a pretty negative view of America to tolerate Ayers and Wright's rhetoric.
Lastly, McCain-Palin needs to point out the media hypocrisy here with an analogy. How would McCain be treated if he had worked for Eric Rudolph or the unabomber?