Is it possible to disable/remove the speech-to-text button on Android in the address bar?

I keep accidentally tapping it when trying to tap at the end of the address to edit it. Currently this is my biggest struggle with Firefox.