It was established to both manufacture firearms and also build naval vessels. The shipyard, plant and machinery were initially leased from Thomas Hunt and Company, an American firm within the concessions of Shanghai. Due to the influx of workers and the reluctance of the concession authorities to allow arms to be manufactured within their territory, the Chinese authorities purchased the plant and equipment and combined these with the existing assets of the old Suzhou and Anqing arsenals as well as new equipment purchased by Yung Wing in the United States to form the new Jiangnan Arsenal in 1865.

The Jiangnan Arsenal was the largest of the arsenals established during the Self-Strengthening Movement, and also the one with the largest budget—from 1869, its annual budget was more than 400,000 silver taels. A series of high officials, including Zeng Guofan, Zuo Zongtang, and Zhang Zhidong, served as its head, although Li Hongzhang served the longest term during the Qing Dynasty. Most of the senior technical staff were Westerners, such as the first chief engineer, American T. F. Falls, and prolific translator John Fryer. During the Tongzhi era, the Arsenal was the largest weapons factory in East Asia. Among its other achievements were the first domestically produced steam boat (the Huiji) in 1868 and the first domestically produced steel in 1891. As well as its manufacturing works, the Arsenal also comprised a language school, a translation house and a technical school. It became forerunner of school training for foreign service civil servants.