Deep learning 二元DenseNet 121分类器仅以概率预测正值>；0.5

我从这个github repo中借用了代码来训练DenseNet-121[

github代码用于CheXpert胸部X光数据集上的14类分类。我已将其修改为二进制分类

# initialize and load the model
pathModel = "/ds2/images/model_ones_2epoch_densenet.tar"#"m-epoch0-07032019-213933.pth.tar"

我初始化14类模型，以便使用预训练权重：

model = DenseNet121(nnClassCount).cuda()

model = torch.nn.DataParallel(model).cuda()
modelCheckpoint = torch.load(pathModel)
model.load_state_dict(modelCheckpoint['state_dict'])

然后转换为二进制分类：

nnClassCount = 1
model.module.densenet121.classifier = nn.Sequential(
    nn.Linear(1024, nnClassCount), 
    nn.Sigmoid()
).cuda()
model = torch.nn.DataParallel(model).cuda()

然后通过：

batch, losst, losse = CheXpertTrainer.train(model, dataLoaderTrain, dataLoaderVal, nnClassCount, 100, timestampLaunch, checkpoint = None, weight_path = weight_path)

我的培训数据以两列csv格式显示，列标题为（“路径”和“类正”），路径位置位于第一列，0或1位于第二列。我在编译培训列表时使用过采样，因此csv中的路径大致在0和1之间分为50/50。无序排列

我使用livelossplot监控培训/验证损失和准确性。我的损失图与预期一样，但精度图在0.5左右平直（如果网络表示其100%为正或负，这对于50/50数据是有意义的）.我假设我在做预测的时候做错了什么，但也许训练中有些地方是不正确的

对于我正在运行的预测和概率：

 varOutput = model(varInput)
    _, preds = torch.max(varOutput, 1)
    print('varshape: ',varOutput.shape)
    probs = torch.sigmoid(varOutput)

*我的问题：pred都是0，prob都在0.5以上*

以下是github的初始代码：

import os
import numpy as np
import time
import sys
import csv
import cv2
import matplotlib.pyplot as plt

import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
import torchvision
import torchvision.transforms as transforms
import torch.optim as optim
import torch.nn.functional as tfunc
from torch.utils.data import Dataset
from torch.utils.data.dataset import random_split
from torch.utils.data import DataLoader
from torch.optim.lr_scheduler import ReduceLROnPlateau
from PIL import Image
import torch.nn.functional as func

from sklearn.metrics.ranking import roc_auc_score
import sklearn.metrics as metrics
import random

use_gpu = torch.cuda.is_available()
# Paths to the files with training, and validation sets.
# Each file contains pairs (path to image, output vector)
pathFileTrain = '../CheXpert-v1.0-small/train.csv'
pathFileValid = '../CheXpert-v1.0-small/valid.csv'

# Neural network parameters:
nnIsTrained = False                 #pre-trained using ImageNet
nnClassCount = 14                   #dimension of the output

# Training settings: batch size, maximum number of epochs
trBatchSize = 64
trMaxEpoch = 3

# Parameters related to image transforms: size of the down-scaled image, cropped image
imgtransResize = (320, 320)
imgtransCrop = 224

# Class names
class_names = ['No Finding', 'Enlarged Cardiomediastinum', 'Cardiomegaly', 'Lung Opacity', 
               'Lung Lesion', 'Edema', 'Consolidation', 'Pneumonia', 'Atelectasis', 'Pneumothorax', 
               'Pleural Effusion', 'Pleural Other', 'Fracture', 'Support Devices']
class CheXpertDataSet(Dataset):
    def __init__(self, image_list_file, transform=None, policy="ones"):
        """
        image_list_file: path to the file containing images with corresponding labels.
        transform: optional transform to be applied on a sample.
        Upolicy: name the policy with regard to the uncertain labels
        """
        image_names = []
        labels = []

        with open(image_list_file, "r") as f:
            csvReader = csv.reader(f)
            next(csvReader, None)
            k=0
            for line in csvReader:
                k+=1
                image_name= line[0]
                label = line[5:]

                for i in range(14):
                    if label[i]:
                        a = float(label[i])
                        if a == 1:
                            label[i] = 1
                        elif a == -1:
                            if policy == "ones":
                                label[i] = 1
                            elif policy == "zeroes":
                                label[i] = 0
                            else:
                                label[i] = 0
                        else:
                            label[i] = 0
                    else:
                        label[i] = 0

                image_names.append('../' + image_name)
                labels.append(label)

        self.image_names = image_names
        self.labels = labels
        self.transform = transform

    def __getitem__(self, index):
        """Take the index of item and returns the image and its labels"""

        image_name = self.image_names[index]
        image = Image.open(image_name).convert('RGB')
        label = self.labels[index]
        if self.transform is not None:
            image = self.transform(image)
        return image, torch.FloatTensor(label)

    def __len__(self):
        return len(self.image_names)
#TRANSFORM DATA

normalize = transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
transformList = []
#transformList.append(transforms.Resize(imgtransCrop))
transformList.append(transforms.RandomResizedCrop(imgtransCrop))
transformList.append(transforms.RandomHorizontalFlip())
transformList.append(transforms.ToTensor())
transformList.append(normalize)      
transformSequence=transforms.Compose(transformList)
#LOAD DATASET

dataset = CheXpertDataSet(pathFileTrain ,transformSequence, policy="ones")
datasetTest, datasetTrain = random_split(dataset, [500, len(dataset) - 500])
datasetValid = CheXpertDataSet(pathFileValid, transformSequence)            
#Problèmes de l'overlapping de patients et du transform identique ?

dataLoaderTrain = DataLoader(dataset=datasetTrain, batch_size=trBatchSize, shuffle=True,  num_workers=24, pin_memory=True)
dataLoaderVal = DataLoader(dataset=datasetValid, batch_size=trBatchSize, shuffle=False, num_workers=24, pin_memory=True)
dataLoaderTest = DataLoader(dataset=datasetTest, num_workers=24, pin_memory=True)
class CheXpertTrainer():

    def train (model, dataLoaderTrain, dataLoaderVal, nnClassCount, trMaxEpoch, launchTimestamp, checkpoint):

        #SETTINGS: OPTIMIZER & SCHEDULER
        optimizer = optim.Adam (model.parameters(), lr=0.0001, betas=(0.9, 0.999), eps=1e-08, weight_decay=1e-5)

        #SETTINGS: LOSS
        loss = torch.nn.BCELoss(size_average = True)

        #LOAD CHECKPOINT 
        if checkpoint != None and use_gpu:
            modelCheckpoint = torch.load(checkpoint)
            model.load_state_dict(modelCheckpoint['state_dict'])
            optimizer.load_state_dict(modelCheckpoint['optimizer'])


        #TRAIN THE NETWORK
        lossMIN = 100000

        for epochID in range(0, trMaxEpoch):

            timestampTime = time.strftime("%H%M%S")
            timestampDate = time.strftime("%d%m%Y")
            timestampSTART = timestampDate + '-' + timestampTime

            batchs, losst, losse = CheXpertTrainer.epochTrain(model, dataLoaderTrain, optimizer, trMaxEpoch, nnClassCount, loss)
            lossVal = CheXpertTrainer.epochVal(model, dataLoaderVal, optimizer, trMaxEpoch, nnClassCount, loss)


            timestampTime = time.strftime("%H%M%S")
            timestampDate = time.strftime("%d%m%Y")
            timestampEND = timestampDate + '-' + timestampTime

            if lossVal < lossMIN:
                lossMIN = lossVal    
                torch.save({'epoch': epochID + 1, 'state_dict': model.state_dict(), 'best_loss': lossMIN, 'optimizer' : optimizer.state_dict()}, 'm-epoch'+str(epochID)+'-' + launchTimestamp + '.pth.tar')
                print ('Epoch [' + str(epochID + 1) + '] [save] [' + timestampEND + '] loss= ' + str(lossVal))
            else:
                print ('Epoch [' + str(epochID + 1) + '] [----] [' + timestampEND + '] loss= ' + str(lossVal))

        return batchs, losst, losse        
    #-------------------------------------------------------------------------------- 

    def epochTrain(model, dataLoader, optimizer, epochMax, classCount, loss):

        batch = []
        losstrain = []
        losseval = []

        model.train()

        for batchID, (varInput, target) in enumerate(dataLoaderTrain):

            varTarget = target.cuda(non_blocking = True)

            #varTarget = target.cuda()         


            varOutput = model(varInput)
            lossvalue = loss(varOutput, varTarget)

            optimizer.zero_grad()
            lossvalue.backward()
            optimizer.step()

            l = lossvalue.item()
            losstrain.append(l)

            if batchID%35==0:
                print(batchID//35, "% batches computed")
                #Fill three arrays to see the evolution of the loss


                batch.append(batchID)

                le = CheXpertTrainer.epochVal(model, dataLoaderVal, optimizer, trMaxEpoch, nnClassCount, loss).item()
                losseval.append(le)

                print(batchID)
                print(l)
                print(le)

        return batch, losstrain, losseval

    #-------------------------------------------------------------------------------- 

    def epochVal(model, dataLoader, optimizer, epochMax, classCount, loss):

        model.eval()

        lossVal = 0
        lossValNorm = 0

        with torch.no_grad():
            for i, (varInput, target) in enumerate(dataLoaderVal):

                target = target.cuda(non_blocking = True)
                varOutput = model(varInput)

                losstensor = loss(varOutput, target)
                lossVal += losstensor
                lossValNorm += 1

        outLoss = lossVal / lossValNorm
        return outLoss


    #--------------------------------------------------------------------------------     

    #---- Computes area under ROC curve 
    #---- dataGT - ground truth data
    #---- dataPRED - predicted data
    #---- classCount - number of classes

    def computeAUROC (dataGT, dataPRED, classCount):

        outAUROC = []

        datanpGT = dataGT.cpu().numpy()
        datanpPRED = dataPRED.cpu().numpy()

        for i in range(classCount):
            try:
                outAUROC.append(roc_auc_score(datanpGT[:, i], datanpPRED[:, i]))
            except ValueError:
                pass
        return outAUROC


    #-------------------------------------------------------------------------------- 


    def test(model, dataLoaderTest, nnClassCount, checkpoint, class_names):   

        cudnn.benchmark = True

        if checkpoint != None and use_gpu:
            modelCheckpoint = torch.load(checkpoint)
            model.load_state_dict(modelCheckpoint['state_dict'])

        if use_gpu:
            outGT = torch.FloatTensor().cuda()
            outPRED = torch.FloatTensor().cuda()
        else:
            outGT = torch.FloatTensor()
            outPRED = torch.FloatTensor()

        model.eval()

        with torch.no_grad():
            for i, (input, target) in enumerate(dataLoaderTest):

                target = target.cuda()
                outGT = torch.cat((outGT, target), 0).cuda()

                bs, c, h, w = input.size()
                varInput = input.view(-1, c, h, w)

                out = model(varInput)
                outPRED = torch.cat((outPRED, out), 0)
        aurocIndividual = CheXpertTrainer.computeAUROC(outGT, outPRED, nnClassCount)
        aurocMean = np.array(aurocIndividual).mean()

        print ('AUROC mean ', aurocMean)

        for i in range (0, len(aurocIndividual)):
            print (class_names[i], ' ', aurocIndividual[i])

        return outGT, outPRED
class DenseNet121(nn.Module):
    """Model modified.
    The architecture of our model is the same as standard DenseNet121
    except the classifier layer which has an additional sigmoid function.
    """
    def __init__(self, out_size):
        super(DenseNet121, self).__init__()
        self.densenet121 = torchvision.models.densenet121(pretrained=True)
        num_ftrs = self.densenet121.classifier.in_features
        self.densenet121.classifier = nn.Sequential(
            nn.Linear(num_ftrs, out_size),
            nn.Sigmoid()
        )

    def forward(self, x):
        x = self.densenet121(x)
        return x
# initialize and load the model
model = DenseNet121(nnClassCount).cuda()
model = torch.nn.DataParallel(model).cuda()
timestampTime = time.strftime("%H%M%S")
timestampDate = time.strftime("%d%m%Y")
timestampLaunch = timestampDate + '-' + timestampTime

batch, losst, losse = CheXpertTrainer.train(model, dataLoaderTrain, dataLoaderVal, nnClassCount, trMaxEpoch, timestampLaunch, checkpoint = None)
print("Model trained")

导入操作系统
将numpy作为np导入
导入时间
导入系统
导入csv
进口cv2
将matplotlib.pyplot作为plt导入
进口火炬
导入torch.nn作为nn
导入torch.backends.cudnn作为cudnn
进口火炬视觉
导入torchvision.transforms作为变换
将torch.optim导入为optim
导入torch.nn.功能为tfunc
从torch.utils.data导入数据集
从torch.utils.data.dataset导入随机分割
从torch.utils.data导入数据加载器
从torch.optim.lr_调度器导入ReducelRon
从PIL导入图像
导入torch.nn.function作为函数
从sklearn.metrics.ranking导入roc_auc_分数
导入sklearn.metrics作为度量
随机输入
使用\u gpu=torch.cuda.u是否可用（）
#指向包含培训集和验证集的文件的路径。
#每个文件都包含对（图像路径、输出向量）
pathFileTrain='../CheXpert-v1.0-small/train.csv'
pathFileValid='../CheXpert-v1.0-small/valid.csv'
#神经网络参数：
NNI=假#使用ImageNet预先训练
nnClassCount=14#输出的维度
#培训设置：批量大小、最大纪元数
trBatchSize=64
trMaxEpoch=3
#与图像变换相关的参数：缩小图像的大小、裁剪图像
imgtransResize=（320320）
imgtransCrop=224
#类名
类别名称=[“无发现”，“心脏纵隔扩大”，“心脏增大”，“肺不透明度”，
“肺损伤”、“水肿”、“实变”、“肺炎”、“肺不张”、“气胸”，
“胸腔积液”、“其他胸腔积液”、“骨折”、“支撑装置”]
类CheXpertDataSet（数据集）：
def uuu init uuu（self，image_list_文件，transform=None，policy=“ones”）：
"""
图像列表文件：包含具有相应标签的图像的文件的路径。
变换：要应用于样本的可选变换。
Upolicy：指定与不确定标签相关的政策
"""
图像_名称=[]
标签=[]
打开（图像列表文件，“r”）作为f：
csvReader=csv.reader（f）
下一个（csvReader，无）
k=0
对于csvReader中的线路：
k+=1
图像名称=行[0]
标签=行[5:]
对于范围（14）内的i：
如果标签[i]：
a=浮动（标签[i]）
如果a==1：
标签[i]=1
elif a==-1：
如果策略==“个”：
标签[i]=1
elif策略==“零”：
标签[i]=0
其他：
标签[i]=0
其他：
标签[i]=0
其他：
标签[i]=0
image\u name.append（“../”+image\u name）
标签。附加（标签）
self.image\u name=图像名称
self.labels=标签
self.transform=transform
定义uu获取项目uu（自身，索引）：
“”“获取项的索引并返回图像及其标签”“”
image\u name=self.image\u name[索引]
image=image.open（image\u name）.convert（'RGB'）
label=self.labels[index]
如果self.transform不是None：
image=self.transform（图像）
返回图像，火炬。浮动张量（标签）
定义（自我）：
返回len（self.image\u名称）
#转换数据
normalize=转换。normalize（[0.485,0.456,0.406]，[0.229,0.224,0.225]）
transformList=[]
#transformList.append（transforms.Resize（imgtranskrop））
transformList.append（transforms.RandomResizedCrop（imgtransCrop））
transformList.append（transforms.RandomHorizontalFlip（））
transformList.append（transforms.ToTensor（））
transformList.append（规范化）
transformSequence=transforms.Compose（transformList）
#加载数据集
dataset=CheXpertDataSet（pathFileTrain、transformSequence、policy=“ones”）
datasetTest，datasetTrain=随机分割（数据集[500，len（数据集）-500]）
datasetValid=CheXpertDataSet（pathFileValid，transformSequence）
#患者重叠和转换识别的问题？
dataLoaderTrain=DataLoader（dataset=datasetTrain，batch\u size=trBatchSize，shuffle=True，num\u workers=24，pin\u memory=True）
dataLoaderVal=DataLoader（dataset=datasetValid，batch\u size=trBatchSize，shuffle=False，num\u workers=24，pin\u memory=True）
dataLoaderTest=DataLoader（dataset=datasetTest，num\u workers=24，pin\u memory=True）
类CheXpertTrainer（）：
def序列（型号、dataLoaderTrain、dataLoaderVal、nnClassCount、trMaxEpoch、启动时间戳、检查点）：
#硒