This is largely system-agnostic, but I'll describe it in d20ish terms. Let's say my character is running away from an ever-expanding pool of acid and needs to jump a chasm to join the other PCs in safety. If I have a +10 to a roll and need to get a 20 or more on a d20+10 roll in order to succeed, then luck plays no more into a reroll than it did in the original roll. Sure, I'm getting a chance to have the luck work out the second time when it didn't work out the first time, but a success on that second roll wouldn't typically feel any different than a success on the first roll - nor would it usually be described differently. If I roll a 5 on the first roll and then succeed with a 16 on the second roll, would any of the other characters (or even my character) say, "Wow, that was lucky!" in response to a success on a luck reroll? Probably not. In game, there is usually nothing to distinguish success on the luck reroll from success on the original roll.
Luck bonuses are even worse here, as you don't even have that second roll to differentiate the effect of an unusual degree of luck.
It is certainly true that, in either of these cases, good attention to description can make a difference. If the second jump roll gets described as me falling a bit short but catching a convenient handhold rather than me just jumping successfully across the chasm, then the effect of luck can be seen in game. In a game where these sorts of successes are consistently described in terms of lucky breaks then, if I have a lot of luck rerolls (maybe because of some feats I took), other characters will tend to see me as lucky (rather than just skillful).
A side note:
All the OSR folks reading this have probably already said... "duh." In old-school play, this sort of thing is typically seen as part of the GM's responsibility. That's not to say that all such GMs are good at it - or view their responsibilities equally... and, really, in such games it is usually the initial die roll where luck is seen as coming into play. I find this situation really interesting, but its a topic for another blog post.
How can we support such descriptions with game mechanics, though? I can think of a couple ways. My current favorite is what I call "pushing your luck." In this mechanical variant, luck bonuses don't apply to standard rolls, and rerolls are special. When you fail a roll that you could have otherwise succeeded in, you can try to push your luck. Roll a d20. On a 20 you succeed due to the intervention of something lucky. On a 1, you hit a patch of bad luck and you not only fail, but suffer some other effect (insert typical botching rules). If you have a luck bonus, that expands your range of success on the d20 roll.
Characters might have a limit on how often they can push their luck. This would depend on genre and tone, but I'd suggest once per game session, plus one for any time they could take a luck reroll (due to feats or whatnot). There might be feats or luck powers that would allow people to push their luck on certain rolls that they wouldn't normally be able to succeed at as well.
Unless your game features a lot of luck bonuses, pushing your luck is fairly balanced. It offers chance at a lucky break, but not without a risk. Moreover, it provides a clear indicator and convenient hook for describing when luck comes into play.