This paper compares the wage structure between the public and private sectors in Japan by using a large microdata set covering public and private sector employees. Rather than comparing overall wage levels, we examine the differences in relative wages by gender, age, education, and region. According to the estimation of wage functions, wage gaps by gender and educational attainment are smaller in the public sector than in private companies. The public sector’s age-wage profile is steeper than that of the private sector. Public sector wages are more compressed; the wages are relatively higher at the lower end of the wage distribution and relatively lower at the higher end. The regional wage differential is smaller in the public sector. As a result, the wage level of public sector workers is relatively higher in rural regions and relatively lower in large metropolitan regions. To ensure the efficient provision of public services, it is inappropriate to compare only average wages. We should carefully observe the differences in wage structure by individual characteristics and by region.