Hungary's Environment Minister Gabor Fodor and Japan's ambassador to Budapest Shinichi Nabekura inked the non-binding framework agreement on Tuesday.
But key details including the number and price of the greenhouse gas emission quotas, known as carbon credits, are still to be negotiated next year.
"The memorandum of understanding is a milestone even by international standards, as until now only the Netherlands and Latvia have signed a similar emissions-trade deal," the Hungarian ministry said in a statement.
It will be the first time that the world's second largest economy has bought carbon credits from another government, although Japanese companies have become major shoppers for credits from foreign firms.
Revenues from carbon credits will be used to fund environmentally-friendly development projects, the Hungarian environment ministry said.
The Kyoto Protocol, the landmark treaty against global warming negotiated in Japan's ancient capital in 1997, requires Japan to cut greenhouse gas emissions between 2008 and 2012 by six percent from fiscal 1990 levels.
Japan has sought a leading role in drafting the successor to the Kyoto Protocol but is itself far from meeting its own obligations as its economy enjoys record expansion from recession in the 1990s.
The Kyoto Protocol allows a company or a country that reduces its carbon-dioxide emissions below the target level to sell the extra reduction as credit to others.