The average size of newly built homes hit an all-time high last year, which may appear to be a positive sign, but a closer look at the numbers would suggest a stark difference between the haves and have-nots.

AP Photo/Keith Srakocic, File

The rich are doing well at the top, while the would-be first-time homebuyers are struggling and hoping for a better economic scenario ahead.

Overall in 2013, the average floor area of new homes in the United States increased 3.7 percent to a record 2,600 square feet, according to the U.S. Census Bureau.

Extremely large houses, 4,000 square feet and above, took up their largest share ever, representing 9 percent of homes built. Large increases were also recorded for homes with more bedrooms, bathrooms and garage space.

Much smaller homes, meanwhile, dropped their share to just 4 percent.

The figures prove few first-time buyers are participating in the housing recovery, according to Stephen Melman, director of economic services for the National Association of Home Builders.

"The lenders are requiring higher down payments - softening a little bit from 20 percent, but still pretty significant - and higher credit scores, in order to qualify for a mortgage to purchase a home," Melman said, noting that just 16 percent of homes were purchased by first-timers last year, compared to approximately 30 percent in years past.

Many younger buyers, according to Melman, are burdened by college loan debt and fighting for fewer job opportunities.

Melman referred to today's housing market as one for the "trade-up buyer," families who are bursting at the seams and need more space, or those who just want to build something special.