TV images show a large amount of mud hitting buildings on a slope in the town of Atami, while people run away.
The two dead are said to have been washed into the sea by the soil masses.
The landslide occurred after several days of heavy rain, which has also led to major problems for train traffic.
The rescue work was in full swing on Saturday in an attempt to find survivors under the remains of the destroyed buildings.
About 25,000 people have been told to move to safer areas, and the authorities have declared a crisis situation in the area. Prime Minister Yoshihide Suga has also summoned the government to an emergency meeting.
The landslides appear to have hit the area several times in quick succession. Buildings were swept over and crushed, and the soil masses also took with them a number of cars.
Atami is located in a popular resort area on the Pacific coast about 100 miles southwest of Tokyo. The area that was affected is called Izusan, and houses both homes, shops and a famous shrine.
— .
