In what both governments hoped was a major step forward, the two countries had agreed in 2015 to a deal created to end a row over Korean "comfort women" forced into sex slavery for Japanese soldiers during the World War II.
Abe made the remarks during a meeting with Moon Hee-sang, a special envoy of South Korean President Moon Jae-in, who was sent to deliver his letter to Abe and discuss summit talks.
Asked about Beijing and Moscow's support for the six previous rounds of United Nations sanctions, Kim said both countries are "close neighbors" who "understand our nuclear projection occurred through the US continued nuclear threat and its hostile policy" toward North Korea.
North Korea tested a longer-range missile last weekend, which experts say was a significant advance for a weapons program that aims at having a nuclear-tipped missile that can strike America.
South Korean Unification Ministry spokesman Lee Duk-haeng told reporters the government's most basic stance is that communication lines between South and North Korea should reopen. While visiting Japanese leaders in Tokyo, US Pacific Command chief Admiral Harry Harris called the North a "clear and unsafe threat", remarking, "combining nuclear warheads with ballistic missile technology in the hands of a volatile leader like Kim Jong-un is a recipe for disaster".
Moon won an election last week campaigning on a more moderate approach toward the North and said after taking office that he wants to pursue dialogue as well as pressure.
China has suggested to visiting South Korean envoys it is willing to work towards easing retaliation for Seoul's decision to deploy THAAD missile defense.
Trump has called for an immediate halt to North Korea's missile and nuclear tests and U.S. Disarmament Ambassador Robert Wood said on Tuesday that China's leverage was key and Beijing could do more.
Speaking on the situation in the Asia-Pacific region, the security chief said that Korean peninsula is on the brink of war and accused the powers of provoking the secretive nation by deploying US warships to the Korean peninsula.
Trump has said "a major, major conflict" with North Korea was possible and all options were on the table but wanted to resolve the crisis diplomatically and peacefully, possibly through the use of economic sanctions.
The UN Security Council North Korea sanctions committee has been meeting with regional blocs at the United Nations to push for enforcement of existing sanctions and assist countries with questions on how to implement the measures.
"The risky behavior by North Korea is not just a threat to the Korean peninsula... it's a threat to China, it's a threat to Russia", Harris told a an academic forum in Tokyo.