Millers triumph over hoodoo
Goals by Adam Le Fondre and Kevin Ellison gave Ronnie Moore's side a 2-0 half-time lead but the minnows upped their game in the second half.
Ryan Ashe reduced the arrears on 63 minutes and Rotherham had to withstand pressure before Drewe Broughton restored a two-goal cushion seven minutes from time.
Ashe thrashed in another effort a minute from time to set up a tense finish.
Moore said: "We made it harder than it should have been. The lads are sat in the dressing room like they have lost so at least there is some feeling that we can be better than that."
Rotherham will host either Rochdale or Luton in the second round on the weekend of November 28.