precision

[swjtulinxi]

have you compared the moiu between your cc.py and the original code on cityscapes ???
becauese i have test your cc.py,and .....

[Serge-weihao]

have you compared the moiu between your cc.py and the original code on cityscapes ???
becauese i have test your cc.py,and .....

I have checked the mIoUs, but I dont know what is the measure of moiu.

[swjtulinxi]

can the miou reach 80％------------------ 原始邮件 ------------------ 发件人: "Serge-weihao"<[email protected]> 发送时间: 2020年1月10日(星期五) 晚上6:06 收件人: "Serge-weihao/CCNet-Pure-Pytorch"<[email protected]>; 抄送: "swjtulinxi"<[email protected]>;"Author"<[email protected]>; 主题: Re: [Serge-weihao/CCNet-Pure-Pytorch] precision (#1) have you compared the moiu between your cc.py and the original code on cityscapes ??? becauese i have test your cc.py,and ..... I have checked the mIoUs, but I dont know what is the measure of moiu. &mdash; You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[Serge-weihao]

can the miou reach 80％------------------ 原始邮件 ------------------

I load the same checkpoint using our pure PyTorch implementation and the official cc_attention, the mIoU results are almost the same, because the calculation is the same. If you have a ckpt with a testing code of 80％ of the original cc_attention, my implementation can get the same results.

[Serge-weihao]

The same CKPT of 30000/60000, using official inplace-abn
Testing with

1. official CUDA cc_attention
   
2. My CC.py
   

[swjtulinxi]

i see ,thank you for yor code,beacuse i can not use the original code, your code can work on my computer,i don't know why.

[swjtulinxi]

the original code is not like yours,i don't understand his code ,for example , _ext. is for what, i can't understand

def _check_contiguous(*args):
if not all([mod is None or mod.is_contiguous() for mod in args]):
raise ValueError("Non-contiguous input")

class CA_Weight(autograd.Function):
@staticmethod
def forward(ctx, t, f):
# Save context
n, c, h, w = t.size()
size = (n, h+w-1, h, w)
weight = torch.zeros(size, dtype=t.dtype, layout=t.layout, device=t.device)

    **_ext**.ca_forward_cuda(t, f, weight)
    
    # Output
    ctx.save_for_backward(t, f)

    return weight

@staticmethod
@once_differentiable
def backward(ctx, dw):
    t, f = ctx.saved_tensors

    dt = torch.zeros_like(t)
    df = torch.zeros_like(f)

    _ext.ca_backward_cuda(dw.contiguous(), t, f, dt, df)

    _check_contiguous(dt, df)

    return dt, df

[Serge-weihao]

the original code is not like yours,i don't understand his code ,for example , _ext. is for what, i can't understand

They use CPP extension with CUDA, so you may have some problems of compatibility(兼容性). The official inplace-abn is also a CUDA implementation.

[Serge-weihao]

What op sys are you using, Linux or Win?

[swjtulinxi]

win

[Serge-weihao]

This repo uses Synchronized-BatchNorm-PyTorch as cross gpus BatchNorm, it costs more gpu memory. I have tested it for batchsize of 4, the result is about 67, so I think the Synchronized-BatchNorm-PyTorch may have some problems or the training hyperparameters are not good for batchsize of 4. I use inplace-abn under Linux, and I will implement inplace-abn using pure PyTorch in the future.

[swjtulinxi]

i just one GPU,so i use the nn.batchnorm2d,

[swjtulinxi]

the paper said ,the batchsize shoud be 12 or higer when we training ,otherwise it will affect the result

[Serge-weihao]

inplace-abn can save some GPU memory for training, but the official CUDA inplace-abn may not support Windows well. If this repo is helpful for you, you can star it.