The COVID-19 pandemic has resulted in calls for an increased integration of community health workers (CHWs) into the health system response. Historically, CHWs can play an important role in ensuring the sustainability of health policy implementation - by addressing social determinants of health and maintaining care for ongoing health problems. Their frontline work, with close contact to populations, places CHWs in a position of increased vulnerability to becoming infected and to being the target of abuse and violence. These vulnerabilities compound underlying problems faced by CHWs, who often come from poor backgrounds, are insufficiently paid and receive inadequate training. Speaking to a scarcity of studies on how CHWs are impacted by the pandemic, this paper conducts a systematic study of CHWs in Brazil. Based on quantitative and qualitative data collected during June and July 2020, it considers perceptions and experiences of CHWs, comparing them with other health professionals. We study the extent to which the pandemic added to existing vulnerabilities and created new problems and imbalances in the work of CHWs. We conclude that COVID-19 led to a deterioration of the working conditions of CHWs, of their relations with other health professionals, and of their ability to carry out their essential work in the public health system.