Having known the rule, but not the actual language it was written in, I had always assumed it was the way the refs called it in that game. Black and white. Unless you or the opponent plays the puck, it's a hand pass.
However, the rulebook does allow for that grey area, a refs discretion which would apply in your in-game scenario. Going by the rules, it probably shouldn't have been called. But, like I said, the refs tend to make it a black and white case and call it as such. Personally, I think that the phrasing in the rulebook should change and just make it black and white. I hate when refs decide games on calls that other refs don't make due to that grey area.