https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_classifier.py#L505-L508 With this code, all parameters are decayed because the condition "parameter_name in no_decay" will never be satisfied. I've made a PR #32 to fix it.