I'm mentioned about a million times before that I can't speak to any other person's experiences in Japan. It's not only about the subjective nature of experience and how feelings and perceptions vary from person to person, but also how we all have different experiences. No one can live the life you live no matter how hard they try. So, when I say that "vacationing is difficult" in Japan, I'm speaking for me, though there were absolutely cultural factors that played into developing that sense and I didn't feel that way when I lived and worked in America and I don't feel that way now that I am back in the U.S..
I should note that Japanese people often take very short vacations because of pressure from peers and employers not to burden others with their work while they are gone. I have never known one Japanese person from among the hundreds I've talked to about this to take all of their vacation days in a given year (or any year ever). The foreigners usually have no compunctions about taking the time off that they are legally entitled to and granted by their companies, but the former president of my former company (not an eikaiwa/language school) fixed this by illegally setting limits on vacation time (7 days maximum, forever) instead of giving the legal mandate (10 days the first year, 1 added day for each additional year).
Besides this job, my only "regular company job" in Japan, I worked at schools and in situations in which "no work" meant "no pay". I never felt like I could comfortably take a vacation when I lived in Japan. The atmosphere was not conducive to it, my employers did their best to stop it, or I felt I'd be losing (a lot of) money if I did take time off. I don't miss feeling like I can never take time off and truly relax.