For the first decade plus in Japan, I didn't file any tax returns at all. I didn't do this in complete defiance of the law. It was an act of pure ignorance. It wasn't until one of my coworkers pointed out to me that it's written right there in my passport that I am obliged to file returns in the U.S. that I learned that I had to do so despite not making any money there. And I've already talked about filing returns in Japan and the confusion and complexity about whether or not I needed to do so there (sometimes I did and sometimes I didn't depending on whether the company did it for me or I had to do it myself).
Fortunately, the I.R.S. didn't get their undies in a wad over my husband and I missing filing returns for a long period of time because we didn't owe them any money anyway. We filed back returns for 3 or 4 years and then started filing regularly after that. Every year, it was "exempt" from all payments because we weren't rich enough to pay to both countries.
I truly do not miss having to file two tax returns, but the truth is that I have so much other paperwork in the U.S. (especially related to car ownership) that it feels as if I still had to do less in Japan. I think at this point that I had it easier there than here when it comes to bureaucracy, but it could really simply be a wash.