so should i take off the quarter panel then put the door on?
No- The quarter panel and rocker are welded on. It sounds like you need to make some adjustments at the hinge where it mounts on the body (not the door). Just adding shims at the door/hinge mount point changes all the geometry in the hinge. I would suggest removing all your shims and trying this:
1 Loosen the hinges at all points, ( i believe there are 2 bolts on the door and three on the body) not sloppy, just enough to be able to move the door around a little on the body.
2 close the door onto the striker and move the door around until the gap on the back edge is about two paint sticks thick (break it in half and you can slide the door right against it without damaging paint) and the gap to the rocker is even front to back.
3 Snug one bolt at each contact point. This should give you a pretty good starting point to fine tune. You'll have a hard time reaching the ones behind the fender, might be easier to pull the fender off.
As a general rule, adjustments at hinge to door are made to move the door in or out from the body (closer to or farther from the dash) and strait up and down. Adjustments to the hinge/body move the door forward/back and angle up and down
Good luck!