From what I remember it wasn’t even during a scene, he was just playing with it.

No, they were going over the scene right before filming. The shot in question was filming down the barrel of the gun, which is why it was pointed in the direction it was.