The BQ Storage API annotates certain types with additional information so that they can be disambiguated. For example, to tell GEOGRAPHY from STRING columns.
We don't currently annotate the types we create when data is initially downloaded from the REST API.
If we make these consistent, the user could register Arrow extension types, which can hook into the default pyarrow -> pandas conversions. See discussion on: #848 (comment)