gradcam.py have a bug

[whl33886]

Error Message:

conv_out should be multiplied by the average of the conv_out_grad in the reference paper. but conv_out be multiplied by the average of the imggrad in your gradcam.py.

What have you tried to solve it?

def get_cam(conv_out_grad, conv_out):
"""Compute CAM. Refer section 3 of https://arxiv.org/abs/1610.02391 for details"""
weights = np.mean(conv_out_grad, axis=(1, 2))
cam = np.ones(conv_out.shape[1:], dtype=np.float32)
for i, w in enumerate(weights):
cam += w * conv_out[i, :, :]

#=============================
# Put this line of code below
#cam = cv2.resize(cam, (imggrad.shape[1], imggrad.shape[2]))
#=============================

cam = np.maximum(cam, 0)
cam = (cam - np.min(cam)) / (np.max(cam) - np.min(cam)) 
cam = np.uint8(cam * 255)
return cam

def visualize(net, preprocessed_img, orig_img, conv_layer_name):
# Returns grad-cam heatmap, guided grad-cam, guided grad-cam saliency
imggrad = get_image_grad(net, preprocessed_img)
conv_out, conv_out_grad = get_conv_out_grad(net, preprocessed_img, conv_layer_name=conv_layer_name)

#===========================
#imggrad replaced by conv_out_grad
cam = get_cam(conv_out_grad, conv_out)
#Add this line
cam = cv2.resize(cam, (imggrad.shape[1], imggrad.shape[2]))
#===================================

ggcam = get_guided_grad_cam(cam, imggrad)
img_ggcam = grad_to_image(ggcam)

img_heatmap = get_img_heatmap(orig_img, cam)

ggcam_gray = to_grayscale(ggcam)
img_ggcam_gray = np.squeeze(grad_to_image(ggcam_gray))

return img_heatmap, img_ggcam, img_ggcam_gray

[sandeep-krishnamurthy]

@indhub - Can you please take a look at this?

[ankkhedia]

@whl33886 seems like a valid bug to me. I will raise a PR for fixing the same.

[ankkhedia]

@whl33886 Apart from the implementation bug, there is also API usage bug and
gradcam_demo.py doesn't work as mentioned in https://github.com/apache/incubator-mxnet/blob/master/example/cnn_visualization/README.md. The PR also addresses the same.

[ankkhedia]

@whl33886 The issue has been fixed now.
@sandeep-krishnamurthy Could you please close the issue as it has been fixed.
@whl33886 Please feel free to reopen in case closed in error.