You can now filter on the onPremisesExtensionAttributes:
https://graph.microsoft.com/v1.0/users?$count=true&$filter=onPremisesExtensionAttributes/extensionAttribute1 eq 'attr1'
Two important points to note:
You need to set the ConsistencyLevel HTTP request header to eventual. Otherwise you’ll get a 400 status code back with the following message Property 'extensionAttribute1' does not exist as a declared property or extension property.
You need to include $count=true even if you don’t care about the count, otherwise you’ll get a 400 status code back with the following message Property 'extensionAttribute1' does not exist as a declared property or extension property.
Source: https://developer.microsoft.com/en-us/office/blogs/microsoft-graph-advanced-queries-for-directory-objects-are-now-generally-available/.