Feel guilty about taking a vacation from work? Most Americans do, a new survey shows. But a New Jersey doctor says never taking a break from the job is bad for employees' health.

Family walking on beach

In an Alamo Rent A Car survey of 1,500 American adults, nearly half claim they've been a victim of "vacation shaming" — feeling a sense of shame or guilt at their workplace for taking time off to go on vacation. Much of that guilt comes from fellow workers.

And sometimes, the vacation shame is too much too handle. More than 20 percent of respondents said feeling shame is at least somewhat likely to keep them from planning or going on a trip.

This paranoia and anxiety among employees is a product of today's competitive, fast-paced work environment, according to Dr. Steven Tobias, director of the Center for Child and Family Development in Morristown.

"Even though we're getting more employment now, I don't think people have quite gotten over the trauma of the recession," Tobias said. "Everybody's more intense."

We're not meant to just work and work and work and work. We need these breaks.

And that's not healthy, Tobias said. His advice is for workers to stand up for themselves and take the break they need, especially if vacation days are part of one's benefits package.

"Instead of taking time off being something that you're embarrassed about or feel guilty or anxious about...taking that time off, having time for yourself, having time for your family, has to become the norm," Tobias said. "We're not meant to just work and work and work and work. We need these breaks."

Even planning a vacation can be a stress reliever, Tobias added. It offers an "end in sight" for workers who feel overwhelmed or stretched thin.

In the Alamo survey, 41 percent of respondents who received paid vacation said they are leaving some of those days on the table. Among that group, two-fifths admitted they left five more vacation days unused in 2015.