Datasheet

Note that on the M0/SAMD based CircuitPython boards, at least, you can also have internal pullups with

INPUT_PULLUP when using external buttons, but the built in buttons require INPUT_PULLDOWN .

Maybe you're setting up your own external button with pullup or pulldown resistor. If you want to turn off the internal

pullup/pulldown just set the pinMode() to plain INPUT