I was born in NYC, but never lived there. Grew up in NJ's Bergen, Union and Morris Counties and worked in Hunterdon county for over 30+ years. I left for Pennsylvania when I was 25 but still commuted to Clinton NJ until I retired. I just couldn't afford living in NJ for what I was making at the time. (you could get 2-3X the house and property in Pa. than you could in NJ. for comparable prices)
But to your point about Taxed to leave, while I was still working there, someone in NJ State government proposed a law that your pension and/or Social Security would be taxed by the State regardless whether you stayed in NJ, or moved to another State. A co-worker/buddy of mine was furious when he heard about this, as he planned to move after he retired from Union county NJ to North Carolina. Fortunately the bill never made it that far to be voted in.
As far as others have posted, there are some nice areas in NJ. My Stepmother who lived on Long Island NY, always pictured NJ as the Secaucus area, but fell in love with the Morris county home and area we lived at.
But, all in all I'm glad I was able to leave and have more 2A freedoms than the surrounding States of Pennsylvania.