The purpose of this paper is to explore the best practice of small medium enterprises (SMEs) from Japan. Japan has been proven to be successful lesson to be followed by Malaysia. Japan’s SMEs began after World War II in 1945. Thus, it has shown that the development process has intricacies and can be studied by Malaysian SMEs, which began in the 1970s due to the revolution of new economic policies. This paper provides the understanding of the development of SMEs between Japan and Malaysia. Further, we identify a gap between SMEs in Malaysia and SMEs in Japan as best practice. The challenges SMEs face might be different in their respective countries—it might be from the perspective of cultural aspects, market environments, and more, but with the initiative measures, pro and cons of both countries’ SMEs can enhance their duty as the backbone of a country’s economy.