Recent research suggests that family background such as parental education and poverty may have different influences on children by gender of the child. In developed countries, the gender gap in the proportion majoring in science and mathematics (STEM) does not necessarily shrink as the overall gender disparity narrows in educational achievement. There are, however, few studies on the specific roles of household background, particularly the difference in investment behavior of money and time, and the role of interaction between cognitive ability and non-cognitive ability. In this paper, using the Japan Child Panel Survey, we first examine the gender differences in the test scores of mathematics and language, preference for these subjects, and educational investment from parents. Next, we analyzed whether the family environment has different effects on these variables for each gender. Our main results are follows. Firstly, we did not find statistically significant gender differences in the average score of mathematical test scores and ratio of top 10% students. Compared to boys, girls did not show statistically significant differences in preferences for mathematics, but did showed statistically significant preferences for languages. Secondly, the income elasticity of expenditure on after school activities were larger to girls, especially of expenditure on artistic lessons.