The Japanese community in Australia was relatively small in the nineteenth century. Until 1866 it was a capital offence for Japanese to leave Japan. However, in the later part of the nineteenth century Japanese had begun to emigrate. Those who went to Australia during the 1880s and 1890s largely worked as crew for Australian pearlers in northern Australia. Others worked in the Queensland sugar cane industry, or were employed in service roles. By 1891, only 30 had settled in Victoria.

The passage of the Immigration Restriction Act in 1901 restricted all non-European immigrants, including the Japanese. However, temporary permits allowed some Japanese immigrants to land in Australia, and by 1904 Japanese immigrants were exempted from the dictation test when applying for extended residency.

When the war against Japan broke out in 1941 the population was almost entirely interned. Most were deported when the war ended. Japanese communities and businesses across the country were effectively eradicated. In Victoria the community size plummeted from 273 people in 1933 to 96 in 1947.

Immigration from Japan remained banned until 1949. During the next five years numbers increased with the arrival of over 500 Japanese war brides. By 1954 the community in Victoria had climbed to 238, and by 1961 had reached 606.

The end of the White Australia policy in 1973 saw more Japan-born businesspeople, students and tourists arrive in Australia. The Japan-born population in Victoria doubled in a decade to 2,744 in 1986, and increased to 4,675 by 2001.

Today Japan-born Victorians are largely employed as professionals, and live predominantly in the more affluent eastern suburbs. The community organises a popular annual Japan Festival in the city of Whitehorse, and its activities are supported through organisations such as the Japan Club of Victoria.