Loading weights without `map_location` with `strict=True` doesnt throw error

Suppose I’m running code with torch.nn.distributed on two GPUs. If the pre-trained weights were saved using cuda:0, then loading the same weights on different GPUs fails without throwing error despite having strict = True checkpoint = torch.load(args.pretrain) checkpoint['model_state_dict']['ln_final.bias'].device # device(type='cuda', index=0) msg = model.load_state_dict(checkpoint['model_state_dict'], strict=True) # always return <All keys matched successfully> If the above process is run...