Sometimes it's your selection of drums that compliments the sample. Maybe your drums are too hard and the sample just sits in the back. Maybe your drums are too soft and the sample just sounds too normal. Maybe your levels need to be tweaked. Maybe your drum pattern and rhythm isn't complimenting the sample. Maybe take out a couple hi-hats, or offset the snare, or take a snare out so it only hits twice per bar. When the rhythm sounds good, maybe change your actual drum sounds and see if that's it.