Calculating the partial differentiation of torch.autograd.grad in the code below will result in None.
I can't see the contents of torch.autograd.grad, so I wonder if there is any problem during the partial differentiation calculation, but I still don't know what's going on.
Also, loss and params.values() used in torch.autograd will be properly printed.
loss example:tensor(0.5828, grad_fn=<MseLossBackward 0>)
Example of param.values(): odict_values([Parameter containing:tensor([-0.1331]], requirements_grad=True),
If anyone knows anything, please let me know.
Thank you for your cooperation.
x=torch.land(4,1)*4*math.pi-2*math.pi
y=torch.sin(x)
optimizer.zero_grad()
params = OrderedDict(model.named_parameters())
for itr range (1000):
pred_y=model(x)
loss=nn.MSEloss(pred_y,y)
grads=torch.autograd.grad(loss,params.values(),create_graph=True,allow_unused=True)
class Net(nn.Module):
def_init__(self, i_channel, o_channel, l_channel):
super(Net,self).__init__()
self.i_net=nn.Linear(i_channel,l_channel)#i_channel=1,l_channel=1
self.l_net=nn.Linear(l_channel,l_channel)#l_channel=1,l_channel=1
self.o_net=nn.Linear(l_channel,o_channel)#l_channel=1,o_channel=1
nn.init.normal_(self.i_net.weight, -1.0, 1.0)
nn.init.normal_(self.l_net.weight, -1.0, 1.0)
nn.init.normal_(self.o_net.weight, -1.0, 1.0)
self.relu=nn.ReLU()
def forward (self, x):
x = self.relu(self.i_net(x))
x = self.relu(self.l_net(x))
x = self.relu(self.o_net(x))
return x
Excuse me.
Fixed the forward part of the second Net class and it was resolved.
Specifically, grad=0.0 was mass-produced by the ReLU function at the end of the output layer, so we removed it.
In other words,
def forward (self, x):
x = self.relu(self.i_net(x))
x = self.relu(self.l_net(x))
x = self.relu(self.o_net(x))
return x
Removed the ReLU in the output layer of the .
def forward (self, x):
x = self.relu(self.i_net(x))
x = self.relu(self.l_net(x))
x = self.o_net(x)
return x
Sorry for the trouble.
I would like to express my appreciation to those involved in the response.
© 2024 OneMinuteCode. All rights reserved.