[LEAD]Back in July, the Sado Island Gold Mine was recognized as a UNESCO World Heritage site, despite controversy of its history of forced labor. The Japanese government held a memorial ceremony for the victims of Korean forced labor, but the Korean government refused to attend. Instead they held its own ceremony on-site with the victims' families. Here's why.[REPORT]The first memorial ceremony dedicated to the Sado mine, where gold and silver were excavated. Soundbytes: A moment of silence.The ceremony was organized to pay homage to all miners who worked at the mine between 1601 and 1989.Japan's Parliamentary Vice-Minister for Foreign Affairs Akiko Ikuina, who attended the ceremony on behalf of the Japanese government, expressed respect and condolences to all workers who worked at the mine over some 380 years.Akiko Ikuina / Parl. vice minister, Japan's foreign ministryI'd like to express my respect to the miners for their hard work, offer my condolences to all the deceased.When the Sado Island Gold Mines were designated as a UNESCO World Heritage site back in July, Japan promised to hold memorial ceremonies because of its history of forced labor.Ikuina acknowledged that Korean workers had to work in harsh conditions at the time.Akiko Ikuina / Parl. vice minister, Japan's foreign ministryThey risked their lives working in harsh conditions.However, the phrase "forced labor" was never mentioned in her speech.Akiko Ikuina / Parl. vice minister, Japan's foreign ministryMany people came from the Korean Peninsula in line with the 1940s policy on wartime workers.Many seats at the ceremony remained empty, as Korea decided not to attend. Korean ambassador to Japan Park Cheol-hee and nine members of the bereaved families arrived on Sado Island after the ceremony.Korean government officials and victims' relatives held their own memorial ceremony on Monday in a building that was used as a dormitory by forced laborers at the time.The foreign ministry said the ceremony was intended to show Korea's firm resolve not to compromise with Japan on such historical issues.