We're feeling worse about the economy. Consumer confidence unexpectedly dipped to its lowest level in nine months.

The Conference Board says its index of consumer attitudes fell to 60.6 from a downwardly revised 65.4 in July. Economists had expected an increase to 66. The index now stands at its lowest point since November 2011 when the reading was at 55.2.

This year, the index declined in January, rose in February and then posted four months of declines before registering an increase in July. August's reading indicates that the gains in the job and housing markets aren't big enough to put to rest Americans' economic fears.

"What we're seeing here is consumer anxiety. Are we going to get fooled yet again?" said Ken Goldstein, Economist at the Conference Board. "In three years, we've been disappointed, thinking that the economy is going to improve a little bit only to find out it didn't happen."

"The concern isn't over where we are right now, but wondering whether this small improvement is here to stay. Is this just another mirage or are things really beginning to improve a little bit because of what's going on the housing market?" said Goldstein. "If that is the case, then consumer confidence may bump up over the next couple of months, which could make for a decent holiday shopping season for retailers. But, if that isn't the case and numbers beginning to falter, consumers will be very disappointed yet again. Basically, they are taking the wait-and-see approach for right now."

"The signal here in August is that consumers are not going to jump on this. There's anxiety about whether it's for real," said Goldstein.

Aside from the housing market, the job market is slowly on the mend as well. Employers added 163,000 jobs in July, the most since February. Job gains averaged 73,000 jobs a month from April through June, but that's not enough to keep up with a rising population. The unemployment rate increased to 8.3 percent from 8.2 percent in June.

Associated Press contributed to this report