An important concern about the surge in the number of refugees arriving in Europe is increased support for far-right, nationalist, anti-immigration parties. This paper studies a natural experiment in an Austrian federal state to identify the causal effect of exposure to refugees in the neighborhood on the support for the far-right Freedom Party of Austria (FPÖ). In the Upper Austria state election in September 2015 the FPÖ doubled its vote share with a fierce anti-asylum campaign. Since only 42 percent of Upper Austrian communities hosted refugees at the time of the election, direct exposure to refugees varied at the local level. To account for the potential endogeneity in the distribution of refugees, I use pre-existing group accommodations as instrumental variable. To cope with the sudden inflow of large numbers of refugees, these buildings were used for refugee accommodation and thus strongly increase the probability of refugee presence in the community. In line with the contact hypothesis I find that hosting refugees in the community dampens the positive overall trend and decreases FPÖ support by 4.42 percentage points in state elections. Further analysis using exit poll data reveals a positive effect on the optimism in the population that the integration of refugees can be managed. Placebo tests show that there were no effects in elections prior to 2015.