distributed train never hit if

[mambo06]

so i run train_dist.py and add print(param) under if section
`

  if type(param) is torch.Tensor:

        print(param)

        dist.all_reduce(param.grad.data, op=dist.reduce_op.SUM, group=0)

`
param never printed which means all_reduce never called.

[TomsyPaul]

I even commented out
# average_gradients(model)
line in run() and there is no difference in output!!

[TomsyPaul]

This line can be commented out, I think.
# if type(param) is torch.Tensor: