Code for Cityscape dataset ?

[LongLong-Jing]

@hfslyc

Thanks for your great work. Did you implement the code to reproduce the experiments in Cityscape dataset in the paper? Thanks!

[watchmexiang]

Hello, I have the same question. Have you trained your data set with it, thank you

[tuscasp]

Have someone managed to train the model in different datasets? How are you calling train.py? I have succeeded to train in the VOC dataset, but whenever I try to do the same in one of my own I get some error messages.

I started keeping the GPU, which is leading to:

Traceback (most recent call last):
  File "train.py", line 506, in <module>
    main()
  File "train.py", line 423, in main
    loss_seg = loss_calc(pred, labels, args.gpu)
  File "train.py", line 173, in loss_calc
    return criterion(pred, label)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/AdvSemiSeg/utils/loss.py", line 34, in forward
    loss = F.cross_entropy(predict, target, weight=weight, size_average=self.size_average)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 2056, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1871, in nll_loss
    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed.  at /opt/conda/conda-bld/pytorch_1556653183467/work/aten/src/THNN/generic/ClassNLLCriterion.c:92

If I omit the GPU, I have this error:

Traceback (most recent call last):
  File "train.py", line 506, in <module>
    main()
  File "train.py", line 423, in main
    loss_seg = loss_calc(pred, labels, args.gpu)
  File "train.py", line 173, in loss_calc
    return criterion(pred, label)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/AdvSemiSeg/utils/loss.py", line 34, in forward
    loss = F.cross_entropy(predict, target, weight=weight, size_average=self.size_average)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 2056, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1871, in nll_loss
    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed.  at /opt/conda/conda-bld/pytorch_1556653183467/work/aten/src/THNN/generic/ClassNLLCriterion.c:92

I have been running the script with

python train.py --snapshot-dir my_snapshots\
                --data-dir dataset/my_dataset \
                --data-list dataset/my_ds_list/my_train_list.txt \
                --input-size 80,80 \
                --num-classes 2 \
                --partial-data 0.125 \
                --batch-size 5 \
                --num-steps 2000 \
                --lambda-adv-pred 0.01 \
                --lambda-semi 0.1 --semi-start 500 --mask-T 0.2 \

As I am working on python3.6, I had to do the following changes in the source code:

• substitute Enumerate.next() by Enumerate.__next__()
• substitute xrange by range

My current file structure is:

AdvSemiSeg >> train.py
                           dataset     >> my_dataset       >> JPEGImages                   >> imgs.jpg
                                                                                 SegmentationClassAug >> masks.png
                                                   my_ds_list        >>  my_train_list
                            my_snapshots
```

[Orchid0714]

@ tuscasp
I have some problems on training the VOC dataset, the error is as follows:
runfile('G:/work_z/AdvSemiSeg-master/train.py', wdir='G:/work_z/AdvSemiSeg-master')
Downloading: "http://vllab1.ucmerced.edu/~whung/adv-semi-seg/resnet101COCO-41f33a49.pth" to C:\Users\Admin/.torch\models\resnet101COCO-41f33a49.pth
168it [00:00, 168012.18it/s]
Traceback (most recent call last):
File "", line 1, in
runfile('G:/work_z/AdvSemiSeg-master/train.py', wdir='G:/work_z/AdvSemiSeg-master')
File "e:\python\python36\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
execfile(filename, namespace)
File "e:\python\python36\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "G:/work_z/AdvSemiSeg-master/train.py", line 473, in
main()
File "G:/work_z/AdvSemiSeg-master/train.py", line 206, in main
saved_state_dict = model_zoo.load_url(args.restore_from)
File "e:\python\python36\lib\site-packages\torch\utils\model_zoo.py", line 66, in load_url
_download_url_to_file(url, cached_file, hash_prefix, progress=progress)
File "e:\python\python36\lib\site-packages\torch\utils\model_zoo.py", line 106, in _download_url_to_file
.format(hash_prefix, digest))
RuntimeError: invalid hash value (expected "41f33a49", got "99a66a06aed2ae4b02ddfb0d43c373ade53336020298039578d6187b1f3e8fc2")

I don't know how to modify it. can you tell me how to settle this? thank you !

[tasveerahmad]

Hello guys,
Recently, I have managed to run the code correctly on Pascal VOC dataset, but when I tried to run the code on my own custom dataset or CityScape dataset, I am unable to run and faced debugging issues. Since I am working on python3.6, I made the following changes in the source code:

• substitute Enumerate.next() by Enumerate.next()
• substitute xrange by range

Also, I change the tensor type from gpu to cpu, e.g. images = Variable(images).cuda(args.gpu) to images = Variable(images).cpu().
As I searched on the internet, I noticed there might be some issue with the number of class. But then I guessed that I have correctly chosen the number of classes, as 30 classes in cityscapes case; or I changed the number of class to 31, but even then facing the same error.
BTW, I'm running the code on GoogleColab, and it worked successfully for Pascal VOC dataset.
The Error report is as below,

/usr/local/lib/python3.7/dist-packages/torch/optim/sgd.py:105: UserWarning: optimizer contains a parameter group with duplicate parameters; in future, this will cause an error; see github.com/pytorch/pytorch/issues/40967 for more information
super(SGD, self).init(params, defaults)
train.py:379: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
D_out = interp(model_D(F.softmax(pred)))
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:1960: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
train.py:382: DeprecationWarning: np.bool is a deprecated alias for the builtin bool. To silence this warning, use bool by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.bool_ here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
ignore_mask_remain = np.zeros(D_out_sigmoid.shape).astype(np.bool)
/usr/local/lib/python3.7/dist-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
pred_train = 88
label_train = 88
Traceback (most recent call last):
File "train.py", line 524, in
main()
File "train.py", line 435, in main
loss_seg = loss_calc(pred, labels)
File "train.py", line 178, in loss_calc
return criterion(pred, label)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/content/drive/MyDrive/Tasweer_Research/Adversarial_Learning_Semi_supervised/AdvSemiSeg/utils/loss.py", line 49, in forward
loss = F.cross_entropy(predict, target, weight=weight, size_average=self.size_average)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py", line 3014, in cross_entropy
return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
IndexError: Target 119 is out of bounds.