The purpose of our paper is to introduce the British Colonial Office records pertaining to Hong Kong, Macao, and the Straits Settlements in the century between the First Opium War and the Second World War (1838 and 1938). We begin with a general introduction to the British Colonial Office records, explaining where they are located, how they are organized, and what they contain. Our focus will be on two collections: (1) Colonial Office, Hong Kong, Original Correspondence (commonly referred to as CO129), and (2) Colonial Office, Straits Settlements, Original Correspondence (commonly referred to as CO273). Because these two archival collections are quite huge, we will limit our scope to descriptions about the Chinese underclass in these port areas, namely coolies, pirates, and members of sworn brotherhoods. Despite that there are few primary sources in the Chinese language on these topics for the period under discussion, the British Colonial Office records are essential documents not only because they can help us fill in the gaps, but also because they provide us with a new perspective for studying China’s history from below. These foreign records reveal much new information about how coolies, pirates, and brotherhoods organized themselves, what roles they played in local society, as well as how the British colonial governments viewed, regulated, and criminalized the Chinese underclass during the heydays of colonialism.