Kleinveldt struck early to get rid of Paul Horton, who made just 12, before the home side pushed on with Neil Dexter (98) leading the way.
Leicestershire had made their way to 155 before the second wicket fell, with Olly Stone, making his first appearance of the season, dismissing Angus Robson for 56.
Northants opener Jake Libby then got in on the act with the ball as he had Mark Cosgrove caught by Kleinveldt for 47.
And then the big South African claimed another scalp of his own as he had Mark Pettini caught by Ben Duckett.
Leicestershire finished the day on 311 for five, witn former Northants man Niall O’Brien ticking along on 15 alongside Ben Raine, who is yet to score.