ERROR conda.core.link:_execute_actions(337): An error occurred while installing package 'defaults::tqdm-4.32.1-py_0'.
CondaError: Cannot link a source that does not exist. C:\ProgramData\Anaconda3\Scripts\conda.exe
Running `conda clean --packages` may resolve your problem.
Attempting to roll back.
# -*- coding: utf-8 -*-# N is batch size, 输入数据的批量大小; D_in is input dimension, 输入数据的维度;# H is hidden dimension， 隐藏层维度; D_out is output dimension， 输出层维度.N,D_in,H,D_out=64,1000,100,10# Create random input and output data 用随机数生成输入数据x、目标数据yx=np.random.randn(N,D_in)y=np.random.randn(N,D_out)# Randomly initialize weights 随机初始化权重参数w1=np.random.randn(D_in,H)w2=np.random.randn(H,D_out)learning_rate=1e-6fortinrange(500):# Forward pass: compute predicted y 向前传播 计算预测值y_predh=x.dot(w1)#矩阵乘法 x:64*1000 w1:1000*100 h:64*100h_relu=np.maximum(h,0)#激活函数ReLU，非线性化y_pred=h_relu.dot(w2)#h_relu:64*100 w2:100*10 y_pred:64*10# Compute and print loss 计算和输出损失loss=np.square(y_pred-y).sum()#误差的平方和print(t,loss)# Backprop to compute gradients of w1 and w2 with respect to loss 反向传播，基于损失函数loss计算参数w1、w2的梯度grad_y_pred=2.0*(y_pred-y)# 损失的梯度， grad_y_pred:64*10grad_w2=h_relu.T.dot(grad_y_pred)# 反向传播至w2，h_relu.T:100*64 grad_w2:100*10grad_h_relu=grad_y_pred.dot(w2.T)# 反向传播至h_relu，w2.T:10*100 grad_h_relu:64*100 grad_h=grad_h_relu.copy()# 反向传播至hgrad_h[h<0]=0# grad_h:64*100grad_w1=x.T.dot(grad_h)# 反向传播至w1 x.T:1000*64 grad_w1:1000*100# Update weights 更新权重w1-=learning_rate*grad_w1w2-=learning_rate*grad_w2
# -*- coding: utf-8 -*-dtype=torch.float#定义tensor的数据类型device=torch.device("cpu")# device = torch.device("cuda:0") # Uncomment this to run on GPU 使用GPU作为计算设备# N is batch size; D_in is input dimension; N为输入数据的批量大小; D_in输入数据维度# H is hidden dimension; D_out is output dimension. H 为隐含层维度; D_out输出层维度N,D_in,H,D_out=64,1000,100,10# Create random input and output data 用随机数生成输入数据x、目标数据yx=torch.randn(N,D_in,device=device,dtype=dtype)# 随机的输入数据y=torch.randn(N,D_out,device=device,dtype=dtype)# 随机的输出数据# Randomly initialize weights 用随机值初始化权重参数w1=torch.randn(D_in,H,device=device,dtype=dtype)# 随机初始化权重w2=torch.randn(H,D_out,device=device,dtype=dtype)learning_rate=1e-6fortinrange(500):# Forward pass: compute predicted y 前向传播，计算预测值y_predh=x.mm(w1)# 相当于numpy中的x.dot(w1)h_relu=h.clamp(min=0)# 相当于np.maximum(h, 0)y_pred=h_relu.mm(w2)# Compute and print loss 计算及打印损失值loss=(y_pred-y).pow(2).sum().item()print(t,loss)# Backprop to compute gradients of w1 and w2 with respect to loss 反向传播，基于损失函数计算参数w1、w2的梯度grad_y_pred=2.0*(y_pred-y)grad_w2=h_relu.t().mm(grad_y_pred)grad_h_relu=grad_y_pred.mm(w2.t())grad_h=grad_h_relu.clone()grad_h[h<0]=0grad_w1=x.t().mm(grad_h)# Update weights using gradient descent 使用梯度下降法更新权重参数w1-=learning_rate*grad_w1w2-=learning_rate*grad_w2
# -*- coding: utf-8 -*-importtorch#定义tensor的数据类型dtype=torch.floatdevice=torch.device("cpu")# device = torch.device("cuda:0") # Uncomment this to run on GPU# N为输入数据的批量大小; D_in输入数据维度;# H 为隐含层维度; D_out输出层维度.N,D_in,H,D_out=64,1000,100,10# 用随机数生成输入数据x、目标数据y的Tensor.# 这两个Tensors在反向传播时，无需进行梯度，故reqires_grad=False（缺省情况）x=torch.randn(N,D_in,device=device,dtype=dtype)y=torch.randn(N,D_out,device=device,dtype=dtype)# 用随机数生成权重参数w1、w2的Tensors # 这两个Tensors在进行反向传播时，需要计算梯度，故设置为requires_grad=True.w1=torch.randn(D_in,H,device=device,dtype=dtype,requires_grad=True)w2=torch.randn(H,D_out,device=device,dtype=dtype,requires_grad=True)learning_rate=1e-6fortinrange(500):# 进行正向传播#计算预测值y_pred.y_pred=x.mm(w1).clamp(min=0).mm(w2)#计算损失值.# 损失值为一个大小为(1,)的Tensor，即为标签。# loss.item() 为损失值.loss=(y_pred-y).pow(2).sum()print(t,loss.item())# Use autograd to compute the backward pass. This call will compute the# gradient of loss with respect to all Tensors with requires_grad=True.# After this call w1.grad and w2.grad will be Tensors holding the gradient# of the loss with respect to w1 and w2 respectively.# 用autograd计算反向传播，这里会根据所有设置了requires_grad=True的Tensor# 计算loss的梯度， w1.grad和w2.grad将会保存loss对于w1和w2的梯度loss.backward()# Manually update weights using gradient descent. Wrap in torch.no_grad()# because weights have requires_grad=True, but we don't need to track this# in autograd.# An alternative way is to operate on weight.data and weight.grad.data.# Recall that tensor.data gives a tensor that shares the storage with# tensor, but doesn't track history.# You can also use torch.optim.SGD to achieve this.# 手动更新weight，需要用torch.no_grad()，因为weight有required_grad=True#但我们不需要在 autograd中跟踪这个操作#torch.autograd.no_grad的作用是在上下文环境中切断梯度计算，在此模式下，#每一步计算结果中requires_grad都是False，即使input设置为quires_grad=Truewithtorch.no_grad():w1-=learning_rate*w1.gradw2-=learning_rate*w2.grad# Manually zero the gradients after updating weights 更新权重后，手工将梯度置零w1.grad.zero_()w2.grad.zero_()
# -*- coding: utf-8 -*-importtorchclassMyReLU(torch.autograd.Function):"""
We can implement our own custom autograd Functions by subclassing
torch.autograd.Function and implementing the forward and backward passes
which operate on Tensors. 通过继承torch.autograd.Function，我们能执行自定义的自动求导函数，而且执行正向和反向传播
"""@staticmethoddefforward(ctx,input):"""
In the forward pass we receive a Tensor containing the input and return
a Tensor containing the output. ctx is a context object that can be used
to stash information for backward computation. You can cache arbitrary
objects for use in the backward pass using the ctx.save_for_backward method.
在正向传递中,我们收到一个包含输入和返回输出的张量, ctx是一个上下文对象,可用于存储反向计算的信息.
您可以使用ctx.save_for_backward方法缓存任意对象以用于反向传播。
"""ctx.save_for_backward(input)returninput.clamp(min=0)@staticmethoddefbackward(ctx,grad_output):"""
In the backward pass we receive a Tensor containing the gradient of the loss
with respect to the output, and we need to compute the gradient of the loss
with respect to the input.
在反向传播中,我们收到一个张量,其中包含相对于输出的损失梯度,
我们需要计算相关于输入的损失函数梯度。
"""input,=ctx.saved_tensorsgrad_input=grad_output.clone()grad_input[input<0]=0returngrad_inputdtype=torch.floatdevice=torch.device("cpu")# device = torch.device("cuda:0") # Uncomment this to run on GPU# N is batch size; D_in is input dimension; N为输入数据的批量大小; D_in输入数据维度# H is hidden dimension; D_out is output dimension. H 为隐含层维度; D_out输出层维度N,D_in,H,D_out=64,1000,100,10# Create random Tensors to hold input and outputs. 用随机值生成输入数据x及目标数据yx=torch.randn(N,D_in,device=device,dtype=dtype)y=torch.randn(N,D_out,device=device,dtype=dtype)# Create random Tensors for weights. 初始化权重参数w1=torch.randn(D_in,H,device=device,dtype=dtype,requires_grad=True)w2=torch.randn(H,D_out,device=device,dtype=dtype,requires_grad=True)learning_rate=1e-6fortinrange(500):# To apply our Function, we use Function.apply method. We alias this as 'relu'. 把类MyReLU构造成一个实例relu，便于像函数一样调用。使用自己定义的自动求导方法relu=MyReLU.apply# Forward pass: compute predicted y using operations; we compute# ReLU using our custom autograd operation. 向前传播，计算预测值y_pred,这里计算ReLU使用我们自定义的autogrady_pred=relu(x.mm(w1)).mm(w2)# Compute and print loss 计算及输出损失值loss=(y_pred-y).pow(2).sum()print(t,loss.item())# Use autograd to compute the backward pass. 使用autograd进行向后传播loss.backward()# Update weights using gradient descent 使用梯度下降法更新权重参数withtorch.no_grad():w1-=learning_rate*w1.gradw2-=learning_rate*w2.grad# Manually zero the gradients after updating weights 更新参数后，需要对梯度置为0w1.grad.zero_()w2.grad.zero_()
# -*- coding: utf-8 -*-importtorch# N is batch size; D_in is input dimension; N为输入数据的批量大小; D_in输入数据维度# H is hidden dimension; D_out is output dimension. H 为隐含层维度; D_out输出层维度N,D_in,H,D_out=64,1000,100,10# Create random Tensors to hold inputs and outputs 创建输入、目标数据的Tensor，随机生成训练数据x=torch.randn(N,D_in)y=torch.randn(N,D_out)# Use the nn package to define our model as a sequence of layers. nn.Sequential# is a Module which contains other Modules, and applies them in sequence to# produce its output. Each Linear Module computes output from input using a# linear function, and holds internal Tensors for its weight and bias.#使用nn包来定义model作为layers 的序列.# nn.Sequential是一个Module，该Module包含其他Modules(如Linear, ReLU等)# Sequential Module会序列化的执行这些 Modules, 並且自动计算其output和grads.#注意因为是序列化执行的, 因此无需自定义 forward. 这是与 nn.Module 的区別之一.# 建立Sequential模型（顺序执行若干步骤）# nn.Sequential是一个包含其他Module的模块，按顺序调用它们来产生输出# 每个Linear模块用线性方程来计算输出，为weight和bias保留中间Tensormodel=torch.nn.Sequential(torch.nn.Linear(D_in,H),torch.nn.ReLU(),torch.nn.Linear(H,D_out),)# The nn package also contains definitions of popular loss functions; in this# case we will use Mean Squared Error (MSE) as our loss function.#定义损失函数，这里用Mean Squared Error (MSE)作为损失函数loss_fn=torch.nn.MSELoss(reduction='sum')# 使用均方误差learning_rate=1e-4fortinrange(500):# Forward pass: compute predicted y by passing x to the model. Module objects# override the __call__ operator so you can call them like functions. When# doing so you pass a Tensor of input data to the Module and it produces# a Tensor of output data.#前向传播，计算预测值y_pred=model(x)# Compute and print loss. We pass Tensors containing the predicted and true# values of y, and the loss function returns a Tensor containing the# loss.# 计算及打印损失值loss=loss_fn(y_pred,y)print(t,loss.item())# Zero the gradients before running the backward pass.# 进行反向传播前，需要对梯度清零model.zero_grad()# Backward pass: compute gradient of the loss with respect to all the learnable# parameters of the model. Internally, the parameters of each Module are stored# in Tensors with requires_grad=True, so this call will compute gradients for# all learnable parameters in the model.#反向传播，根据model参数计算损失函数的梯度#每个model的 parameters 存放在含requires_grad=True标签的Tensors中loss.backward()# Update the weights using gradient descent. Each parameter is a Tensor, so# we can access its gradients like we did before.# 利用梯度下降法更新权重参数 .每一个参数都是一个Tensor,并可获取他们的梯度withtorch.no_grad():forparaminmodel.parameters():param-=learning_rate*param.grad# 手动更新参数
# -*- coding: utf-8 -*-importtorch# N is batch size; D_in is input dimension;# H is hidden dimension; D_out is output dimension.# N为输入数据的批量大小; D_in输入数据维度;# H 为隐含层维度; D_out输出层维度N,D_in,H,D_out=64,1000,100,10# Create random Tensors to hold inputs and outputs# 创建随机数据x和目标数据yx=torch.randn(N,D_in)y=torch.randn(N,D_out)# Use the nn package to define our model and loss function.#使用nn包定义model及损失函数#建立Sequential模型（顺序执行若干步骤）model=torch.nn.Sequential(torch.nn.Linear(D_in,H),torch.nn.ReLU(),torch.nn.Linear(H,D_out),)loss_fn=torch.nn.MSELoss(reduction='sum')# Use the optim package to define an Optimizer that will update the weights of# the model for us. Here we will use Adam; the optim package contains many other# optimization algoriths. The first argument to the Adam constructor tells the# optimizer which Tensors it should update.#使用optim包定义更新模型参数的优化器. 我们选择Adam优化器; optim package#还包含其他优化算法. Adam的第一个参数为需要更新的Tensorslearning_rate=1e-4optimizer=torch.optim.Adam(model.parameters(),lr=learning_rate)fortinrange(500):# Forward pass: compute predicted y by passing x to the model.# 计算预测值y_pred=model(x)# Compute and print loss.# 计算打印损失值loss=loss_fn(y_pred,y)print(t,loss.item())# Before the backward pass, use the optimizer object to zero all of the# gradients for the variables it will update (which are the learnable# weights of the model). This is because by default, gradients are# accumulated in buffers( i.e, not overwritten) whenever .backward()# is called. Checkout docs of torch.autograd.backward for more details.# 反向传播前, 使用优化器对所有需要更新参数的梯度清零#这是因为缺省情况下，梯度会存放在缓存里optimizer.zero_grad()# Backward pass: compute gradient of the loss with respect to model# parameters# 反向传播loss.backward()# Calling the step function on an Optimizer makes an update to its# parameters#更新参数optimizer.step()
# -*- coding: utf-8 -*-importtorchclassTwoLayerNet(torch.nn.Module):def__init__(self,D_in,H,D_out):"""
In the constructor we instantiate two nn.Linear modules and assign them as
member variables.
通常我们將具有引用学习参数的层放在__init__函式中, 將不具有引用学习参数的操作放在forward中
"""super(TwoLayerNet,self).__init__()self.linear1=torch.nn.Linear(D_in,H)self.linear2=torch.nn.Linear(H,D_out)defforward(self,x):"""
In the forward function we accept a Tensor of input data and we must return
a Tensor of output data. We can use Modules defined in the constructor as
well as arbitrary operators on Tensors.
"""h_relu=self.linear1(x).clamp(min=0)y_pred=self.linear2(h_relu)returny_pred# N is batch size; D_in is input dimension;# H is hidden dimension; D_out is output dimension.# N为输入数据的批量大小; D_in输入数据维度;# H 为隐含层维度; D_out输出层维度.N,D_in,H,D_out=64,1000,100,10# Create random Tensors to hold inputs and outputsx=torch.randn(N,D_in)y=torch.randn(N,D_out)# Construct our model by instantiating the class defined above# 实例化类，构建模型# 使用自己建立的TwoLayerNet类来搭建网络model=TwoLayerNet(D_in,H,D_out)# Construct our loss function and an Optimizer. The call to model.parameters()# in the SGD constructor will contain the learnable parameters of the two# nn.Linear modules which are members of the model.#构建损失函数及优化器. 其中model.parameters()包含学习参数criterion=torch.nn.MSELoss(reduction='sum')optimizer=torch.optim.SGD(model.parameters(),lr=1e-4)fortinrange(500):# Forward pass: Compute predicted y by passing x to the model# 传入x，计算预测值y_pred=model(x)# Compute and print lossloss=criterion(y_pred,y)print(t,loss.item())# Zero gradients, perform a backward pass, and update the weights.# 对梯度清零，执行反向传播并更新参数optimizer.zero_grad()loss.backward()optimizer.step()
# -*- coding: utf-8 -*-importrandomimporttorchclassDynamicNet(torch.nn.Module):def__init__(self,D_in,H,D_out):"""
In the constructor we construct three nn.Linear instances that we will use
in the forward pass.
实现三个 nn.Linear 层
"""super(DynamicNet,self).__init__()self.input_linear=torch.nn.Linear(D_in,H)self.middle_linear=torch.nn.Linear(H,H)self.output_linear=torch.nn.Linear(H,D_out)defforward(self,x):"""
For the forward pass of the model, we randomly choose either 0, 1, 2, or 3
and reuse the middle_linear Module that many times to compute hidden layer
representations.
Since each forward pass builds a dynamic computation graph, we can use normal
Python control-flow operators like loops or conditional statements when
defining the forward pass of the model.
Here we also see that it is perfectly safe to reuse the same Module many
times when defining a computational graph. This is a big improvement from Lua
Torch, where each Module could be used only once.
在PyTorch中, 我们可以通过for循环来随机的选择中间层的层数, 使得每一次
执行forward函式时, 都有不同的中间层层数. 而这些中间层都来自于同一个Module例項, 因而具有共享的权重参数
"""h_relu=self.input_linear(x).clamp(min=0)for_inrange(random.randint(0,3)):h_relu=self.middle_linear(h_relu).clamp(min=0)y_pred=self.output_linear(h_relu)returny_pred# N is batch size; D_in is input dimension;# H is hidden dimension; D_out is output dimension.# N为输入数据的批量大小; D_in输入数据维度;# H 为隐含层维度; D_out输出层维度.N,D_in,H,D_out=64,1000,100,10# Create random Tensors to hold inputs and outputsx=torch.randn(N,D_in)y=torch.randn(N,D_out)# Construct our model by instantiating the class defined abovemodel=DynamicNet(D_in,H,D_out)# Construct our loss function and an Optimizer. Training this strange model with# vanilla stochastic gradient descent is tough, so we use momentumcriterion=torch.nn.MSELoss(reduction='sum')optimizer=torch.optim.SGD(model.parameters(),lr=1e-4,momentum=0.9)fortinrange(500):# Forward pass: Compute predicted y by passing x to the modely_pred=model(x)# Compute and print lossloss=criterion(y_pred,y)print(t,loss.item())# Zero gradients, perform a backward pass, and update the weights.optimizer.zero_grad()loss.backward()optimizer.step()
