Evaluating using My Own Code

[hmdolatabadi]

Hi!

Thanks for the nice work.
I want to use the three approaches used here for comparison in my own pipeline.
What I do is that

1. Train a model on my poisoned data.
2. Evaluate the feature space representation of the training data using the trained model (ResNet-32, so it would be the output of the 'layer3' below).

   spectre-defense/resnet.py

   Line 112 in ccf594a

   self.layer3 = self._make_layer(block, 64, num_blocks[2], stride=2)

   .
3. Then, I use your Julia code to compute the samples that need to be removed for the target label.

Am I right? Because I am getting a mixed performance and I want to double check.

[jhayase]

In our paper, we actually save the representations one basic block before layer3. (This is layer 14 in the SequentialImageNetwork.) We are very interested in backdoors that can bypass SPECTRE so if you cannot find a layer that works well, we would be happy to look more closely at why SPECTRE is failing.

[hmdolatabadi]

Thanks! I am running my experiments with a slightly modified version of ResNet-32, where the BatchNorm2d is done before the shortcut, and I am not sure whether this is the culprit. Interestingly in this case, Spectre works accurately on label consistent attacks and detect all the poisoned data. But on sinusoid attack and badnets it can remove 84% and 87% of the poisoned data, but the remaining 16% and 13% are enough to poison the model after retraining.

Below is the architecture of the blocks that I am using:

(3): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (shortcut): Sequential()
    )

And here is the blocks in the Spectre code:

(14): BasicBlock(
    (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (shortcut): Sequential()
  )

[jhayase]

Unfortunately, it's not clear to me what effect the different basic block structures might have on the representations. What you can try is pulling representations from various points inside the basic block. This is the idea behind the class BasicBlockSplitter in model.py, although we didn't need to use it in the end for our own experiments.

What you can also try is looking at a PCA pairplot of the representations and see if there is any obvious way to separate the poison and clean points. This lets you determine whether the failure to remove the poison was because

1. the representations of poison data have no detectable spectral signature, or
2. the covariance estimation and QUE scoring failed to detect the spectral signature of the poison.