Abstract

The educational goal of Hiroshima Prefectural University (henceforth, HU) states that first-year EFL students are to achieve CEFR level B2 (the English level of the majority of students was CEFR B1 at the beginning of the semester) upon completion of a three-month intensive English program (henceforth, IEP). Through an active learning instructional approach students are expected to develop the requisite language skills in addition to a set of key competencies fostered through the cooperative learning environment. The aim of the program is for students to improve their language proficiency in preparation for all-English content classes commencing in the subsequent semester. Active learning is defined by HU as a learner-focused approach in which core elements of collaborative, cooperative, and problem-based learning are developed. Adoption of the approach signifies a transition away from traditional knowledge transmission-based to teaching that more actively engages students in the learning process leading to more long-lasting, meaningful learning [61]. However, analysis of results from post testing show only moderate improvement in mean test scores as measured on the TOEFL ibt test. This paper critically examines expectations of the program, both in terms of language improvement and also for evidence of the effectiveness of active learning among Japanese EFL learners

1. Introduction

The educational goal of Hiroshima Prefectural University (henceforth, HU) states that first-year EFL students are to achieve CEFR level B2^{1} upon completion of a three-month intensive English program (henceforth, IEP). Through an active learning instructional approach students are expected to develop the requisite language skills in addition to a set of key competencies fostered through the cooperative learning environment. Active learning is defined by HU as a learner-focused approach in which core elements of collaborative, cooperative, and problem-based learning are developed. Adoption of the approach signifies a transition away from traditional knowledge transmission-based to teaching that more actively engages students in the learning process leading to more long-lasting, meaningful learning ¹.

2. Background

For countries without the experience of colonialization or EFL learning, practical English communicative ability (rather than as an academic subject) has been intensely promoted by both business and governments. In the case of Japan, under the 2013 Second Basic Plan for the Promotion of Education ² since 2017 at least half of all 16-year-olds upon completion of secondary education are expected to have achieved a level of English proficiency equivalent to Grade 3 on the Eiken Test in Practical English Proficiency^{2}. Likewise, half of all high school students are supposed to have acquired English proficiency equivalent to Grade 2 or Grade Pre-2^{3} by the time they matriculate^{4}. However, only 42.6% of middle school students in their final year actually achieved this target in 2017 (the figure was 40.2% for high school graduates). By skill, only 33% of middle school students (12% for high school graduates) were proficient enough to pass the speaking section of the respective test ². A seemingly continual failure in achieving government language targets has resulted in the focus of language instruction and the traditional grammar-translation methodology it encapsulates attracting increasing domestic criticism ³. The perception of ineffectiveness is reinforced by international test-takers’ tables listing individual country’s English test achievements (see Table 1). Furthermore, failure to develop the requisite communicative language ability has been extensively highlighted (‘Why do Japanese have trouble learning English?’; ‘Why Japan's English education is a fiasco’, Japan Times 2017 & 2019, are frequent headlines in English-Japanese newspapers) and a source of continual national chagrin.

3. Active Learning’s Educational Goals

Presenting alternatives to traditional, one-way transmission of knowledge, MEXT has been advocating diversification in teaching methodologies away from the rote memorization, lecture-style teaching for which the Japanese classroom is renowned. The aim has been the promotion of outcome-oriented education methods through meaningful, learning activities in helping learners go beyond reproductive, surface learning to deeper levels of analysis and personalized sense-making ⁵. Specifically, the cultivation of key competencies (e.g., critical thinking, problem-solving ability, communication skills) that remain unutilized during conventional passive learning ⁶. One teaching approach that claims to satisfy the above requirements is active learning, a conceptual framework representing a central feature evident in several educational approaches including learner-centered approaches, cooperative learning, collaborative learning, problem-based learning, and inquiry-based learning. The instructional approach monitors and guides learning based on contemporary definitions synthesized into the following six dimensions: active, interactive, cognitively engaging, emotionally involving, individualized, and independent ⁷. Guided by constructivist learning theory the approach aims to develop understanding over memorization as concepts become transferable and retained through the discovery of drawing meaning from experience in the self-construction of knowledge ⁸. Learning is enhanced through social interaction by actively processing information to discover and construct solutions to problems.

Integrating the above theoretical principles into appropriate and meaningful methods within the classroom, HU’s instructional approach similarly promotes an active learning environment in which collaboration, participation, and reflection feature prominently. Specific key competencies of foresight (conceptual thinking), strategy (logical thinking), global collaboration (communication skills), self-improvement (life-long learning), and energetic drive (leadership) are promoted through a learner-centered curricula that allocates only 20% of class time to one-sided lectures by faculty ¹¹. The remaining 80% are focused on learner-centered activities that aim to develop a deeper understanding of the subject matter. The learning objectives are reflected in a standardized grading rubric that assesses not only measurable academic achievement but also the degree to which the above non-testable competencies have been demonstrated. In regards to language instruction, the learning process is viewed as inextricably linked to the completion of goal-orientated, meaning-focused activities in which all four conditions (exposure, motivation, real language, and a focus on form) needed to effectively learn a language are met ⁹. Real-world tasks aim at developing an understanding of the relationship between form and function ¹⁰ through being more cognitively engaging with a learning process of active, mental and often physical engagement. However, acknowledgement ¹¹ is made of the potential for subjective evaluation of personal qualities and personalities in the name of competency, and the guidelines warn against conflating the assessable with what is valuable. Although this acknowledges pre-defined measurement tools will not always capture everything, regardless of assessment it recognizes the unpredictability of unstructured learning as an environment in which the most meaningful learning occurs.

For individual language skills, the curriculum for speaking instruction views fluency as the basis for linguistic accuracy, with the type of interaction produced regarded as being more productive to language development than the actual form of language used. The learning environment affords students the linguistic freedom to choose from their existing resource to focus a broader range of lexicon on the functional purposes for which the language must be used, rather than ‘bits and pieces of language’ in isolation ⁸. Activities include role playing, demonstrations, and peer teaching. For reading skills, activities include discussions on interpretations of text, summarizing content in groups, and reflecting on convergent-divergent tasks. Listening activities incorporate the latest means by which information is promulgated and provide motivation to be able to listen to popular influencers in English. This includes transcribing the material and summarizing online podcasts, TED talks, YouTube channels etc. Writing activities similarly incorporate the aforementioned social media as a means of contextualizing language and include deconstructing/co-constructing texts and identifying the structure and language particular to each platform. The activities represent a move away from implicit, process writing to a focus on the social purpose of writing and the audience for whom it is intended.

Definitionally, active learning’s inclusion of any instructional methods that engage learners in the learning process (i.e., being involved in “doing” and ‘thinking” about things: ¹² leads to an inexactness of the specific learning goals being promoted. A focus on techniques for instruction rather than learning outcomes fails to specify the degree to which the diverse possibility of higher-order, incidental learning outcomes (i.e., problem-solving, critical thinking ability) beyond standardized academic achievement are being developed. Data for these competencies, whose assessment is more complicated and often lacks a dominant core element, is often less frequently available. Even within the active learning approach, emphasis placed on varying elements of learning means that some instructional techniques are more effective for promoting academic achievement while others influence alternative elements. To illustrate the variance, pure active learning (sometimes referred to as inquiry-based learning) involves students engaging with materials with minimal instructor involvement. Research has highlighted how this type of learning is inferior to guided learning in gaining subject matter knowledge ¹³, ¹⁴. In Instances where active learning does show improved retention of the subject matter instruction and guidance with the activities was provided ^{15, 16, 17}.

4. Empirical Studies Into Active Learning among Japanese EFL learners

Evaluating the effectiveness of active learning within the Japanese setting similarly requires assessment of a range of learning outcomes, careful interpretation of data and the instructional methods employed. Despite differing terminology, analytical frameworks, and methodological designs, similarities allow for commonality to be extracted in allowing comparison within second language instruction. Since 2004, twelve major empirical studies have been conducted into the pedagogical effects of active learning instruction among Japanese EFL learners (see Table 2). All studies were conducted in an EFL context (all carried out in Japan), with the number of participants (predominantly university students, average age: approximately 20) varying from 6 to 112 (average number of participants: 47). Among the studies, a majority reported only positive findings to active learning in terms of improved language performance, while four studies reported mixed results. The remaining study reported unconfirmed influences on learner language performance as a result of active learning instruction. It should be noted that none of the studies attempted to measure development of higher skills that are fostered as a result of a collaborative learning environment. Recommendations in three of the studies advocate active learning instruction as a pedagogical tool in second language education in Japan. It must be acknowledged, however, that any positive effects could be due to temporal, meta-cognitively controllable knowledge rather than meaningful progress in the learners’ interlanguage system.

Among the studies, specific learning outcomes include: examination of pre-/posttest listening data found significantly improved performance in classes with substantial use of interactive-engagement methods ¹⁸. Test scores measuring conceptual understanding were high in classes promoting engagement than in traditional courses. Others show that improved learning gains in critical thinking are link to the nature of active engagement and not to extra time spent on a given topic ^{19, 20}. In terms of speaking performance, ²¹ found significant improvement in the quality of student responses in addition to their involvement in the learning process. In contrast to these findings, ^{22, 23}, and ²⁴ found mixed results for speaking performance. A majority of above studies provide support for active engagement methods, particularly for addressing students’ involvement in the learning process; an essential element of effective teaching. For example, how working in small groups has a positive effect both motivationally and academically ²⁵. The evidence highlighted above supporting and validating the effectiveness of learner engagement (although not overwhelming) on a broad range of language learning outcomes.

5. Discussion

Despite the stated goals and pedagogical justification put forward for active learning by the University, academic achievement has not been transferred in terms of actual language improvement (as measured on the TOEFL test). Even though the intensive English course achieved a 95% completion rate, it was noted that the 30% allocation from competencies inflated grades for those students who might otherwise have been borderline. Feedback from faculty members highlighted the prominent use of Japanese during class, and a reliance on their first language in communicating for non-task activities. This highlights how the success (in terms of language improvement and developing relevant competencies) of active learning may be dependent on several factors not always evident in a collectivist country such as Japan. Relevant cognitive and socio-cultural factors could potentially reduce the effectiveness of the approach in the Japanese setting. It must be recognized that although these factors, or lack of them, may also influence other teaching methodologies, the scope of this paper is restricted to an assessment of active learning only and therefore does not hypothesize about their general effectiveness. The factors are:

1. Socio-cultural factors

2. Existential competencies

3. The active learning format

After addressing each of these factors, suggestions are proposed which, if incorporated into the active learning approach, could help to overcome the highlighted shortcomings. These suggestions address the approach only in relevance to the Japanese setting and are merely suggestions rather than the endorsement of an alternative teaching methodology.

The classroom will not always be seen as a meeting place between student expectation, curricular content, and pedagogical appropriateness ²⁹ due to differing teacher/student beliefs regarding classroom instruction ³⁰. As mentioned, the teacher-centered nature of the Japanese education system shapes and maintains students’ beliefs and concepts they hold in regard to the language learning process. These expectations are recognized as a potentially significant element when making the transition to the apparent ‘randomness’ of active learning ³¹. Instances of student frustration or dissatisfaction are likely to surface when they are tasked to perform independent, creative, autonomous activities; instructional activities inconsistent with preconceived beliefs about learning ³². When students become aware of this discrepancy between expectations and what is actually happening, if unfulfilled, they may result in what are termed ‘mismatches’ ³³. These false assumptions and prejudices which underlie their attitude towards their role in learning must be addressed through the learner’s subjective needs and perception in order to minimize any resistance to a new teaching approach ³⁰. This involvement in the collaborative process of individual cognitive and learning style preferences is therefore an essential element of sensitizing students to both the attitudinal and behavioral expectations required as they move from a teacher-centered system to one where they as learners exercise the power. At the same time, it is important that foreign faculty are also sensitive to specific Japanese cultural traditions both inside and outside the classroom ³⁴.

Even within the classroom setting the situation is not only determined by cognitive, affective and language proficiency factors but also a social dimension ³⁵ has to be considered. The context of the learning situation and the cultural values of the learners’ society can be expected to have a strong influence on participation. In some measure this reflects the Japanese school system’s emphasis on the evaluation paradigm and thereby furnishes additional language anxiety regarding evaluation of subjects’ performance by others ³⁶. This is one reason put forward for the predominance of convergent thinking ³⁷ and the focus on the production of a single ‘right’ answer in second language situations.

These characteristics are products of ‘centralized curricula, didactic and expository teaching styles, concentration on knowledge acquisition, examinations emphasizing reproductive knowledge over genuine thinking’ ¹⁹. While some researchers have attempted to negate the significance of these sociocultural influences, factors that are presented as mitigating these are themselves culturally bound phenomenon (e.g., student reticence, low confidence levels, and fear of making mistakes). For example, how a culture that prizes individual competition organizes its educational system around competitive tasks, resultantly, successful language learners prefer strategies that allow them to work alone rather than social strategies that call for collaboration with others ³⁸. Others has discussed the influence of discoursal factors on interaction and showed it can vary between individual learners coming from different cultural and educational backgrounds ³⁹. These can be attributed to their mother tongue, learning style interference, as well as educational and cultural background ⁴⁰.

Existential competences: selfhood factors linked to the personal traits of identity (i.e., attitudes, values, beliefs) affect communicative interaction ⁴¹ and represent constituents of the user’s emotional, affective, cognitive and social attitudes. As is evident from the lengths of pauses (lexicalized and non-lexicalized) in classroom interactions, the extent to which fear of making mistakes ⁴² or the “shyness factor” appears to determine the level of risk subjects are prepared to undertake when communicating. From the author’s personal observation, during normal communicative lessons this feature can often be observed with Japanese students’ high tendency to write out prepared answers (even for English communication classes) rather than mentally constructing ideas simultaneously as the conversation progresses. A common justification for this practice is that is allows for schema and internal representations to be formed without the constraints of construction while interacting. As a result of this exacting attention to detail ^{42, 43}, terms such exactitude ambiguity avoidance) extended non-lexicalized pauses combined with Japanese culture’s greater tolerance of silence ^{44, 45} may contribute to the lengthening of interruptions while planning takes place. This use of silence may be the result of rule-conflict between English and Japanese conversation style; when confronted with questions students are unable to answer a common recourse is to resort to silence as a face-saving measure and/or convey unfamiliarity by remaining silent ⁴⁶. The dynamics of this conflict seem contributive to uncomfortable silences that are a common feature during communicative activates and one of the most challenging aspects of interaction in the classroom for non-Japanese language teachers ^{47, 48, 49}. Again, from personal observation, explicitly stating lack of familiarity carries a different connotation for a Japanese as it does to Westerners as such an admission of ignorance or unfamiliarity often presupposes insufficient subject knowledge due to lack of intelligence or interest and pertains to the dimension of face and credibility. Similarly related to the confined responses mentioned earlier, under-elaboration also frequently impairs exchanges by restricting subjects’ use of vocabulary and constructs. Students negatively transfer their high-context communication style ^{50, 51, 52} that neither requires everything to be expressed verbally nor appreciates verbosity, but emphasizes ‘accurately expressed thoughtful comments’ ⁴⁶ and leads to the production of linguistically and propositionally accurate minimal utterances. Within high-context communication most of the information is either in the physical context or internalized in the person and therefore not all information needs to be explicitly expressed verbally. When communicating in this environment students expect their interlocutor to have a degree of familiarity so that the need for specifics becomes redundant ⁵³. In contrast, such utterances can often frustrate ⁵⁴ Western EFL teachers from a low-context communication culture (in the author’s case the UK) where information must be elaborated, clearly communicated and highly specific ⁵⁵. Even linguistically advanced Japanese learners can exhibit lack of pragmatic awareness of elaboration ⁵⁶ resulting in under-elaborated answers to seemingly routine everyday questions. In learning terms this represents a failure to recognize the active learning interactions as invitations to speak on a topic and unfortunately lead to quite literal, restricted responses. The above circumscription of responses denotes a psychological process derived from cognitive constraints on learnability and expressibility ⁵⁷ that operate at phonological, syntactic, lexical and pragmatic levels of language use. These types of response imply a reluctance and unpreparedness to expand on linguistic resources and reveals Japanese learners to be less explicit, for reasons stated above, in giving explanatory response for their verbal behavior ⁵⁸. Within the present setting there prevails a strong stereotypical view of the communicative style of Japanese EFL leaners that is supported by in-class observation and personal anecdotal evidence. It contrasts with the Western style of conducting conversations as a kind of ‘logical game’ involving continuous negative and positive judgements and reveals the Japanese preference for conversation that stresses intuitive understanding and mutuality of feeling in social interaction ⁵⁸. The distinct difference could help to illustrate common frustrations among foreign faculty teaching here that students ‘don’t talk enough’ ⁵⁹ when provided with active learning’s communicative tasks in the EFL classroom.

As touched upon in the previous section the speaking style of Japanese can also limit the amount of language production being practiced in the active learning classroom. ⁶⁰ highlights how the Japanese follow a listener talk approach to conversation which tends to value reticence and orderly turn taking with less inclination to dominate conversations. With this style of interaction, the onus is on the listener to understand what is being said rather than seek clarification, in contrast to the type of dialogue which most Western teachers envisage within the EFL classroom; of students engaged in continual interaction as an information sharing activity to practice and develop communicative language proficiency. Furthermore, manifestations of the Japanese cultural concept of Wa or harmony additionally influence intercultural communication through indirect and suggestive comments rather than more direct and explicit statements ^{48, 61} As mentioned above, according to ⁶² Japanese tend to view such ambiguous behavior as perceptive and intelligent while in contrast Western EFL teachers generally view ambiguity as illogical, irrational, and even dishonest or unintelligent. This can be observed when required to provide more detailed information which many students view as irrelevant or superfluous. As a result, asking to provide reasons for choices or preferences, attempting to elicit more detailed accounts can prove problematic and for subjects confronted with such ‘Why…’ questions may even become ambiguous if they feel that their interlocutor is disagreeing with them ⁶².

Attempts to address linguistic skills and competence by distinguishing between ‘convergent,’ and ‘divergent’ tasks seeks to address concerns that task completion is limited to a narrow linguistic achievement ⁶³. The purpose of divergent active learning tasks is to facilitate task completion by incorporating discussion and reasoned arguing at a higher level of language learning. However, if the primary function is ‘to facilitate the unfolding of the learner’s powerful internal syllabus’ ⁶⁴, it would seem to be contradicted by personal observation that much of the negotiation fails to be take place in L2. Students distinguish between the tasks, which are occasionally conducted in L2, and negotiation of meaning, in L1. Therefore, the first objective of using tasks as a means of encouraging a higher frequency of L2 is not being achieved. Although this observation only recognizes active participation, the author accepts that students may be learning from observing but had not yet reached a level where they felt comfortable speaking.

The lack of meaningful communicative opportunity can affect not only linguistic ability but more lastingly motivation. A lack of perceived language improvement can be seen with many Japanese university students regarding themselves as beginners despite seven years of instruction. As a result, they display an ‘apathetic attitude’ ⁶⁵ which often leads to a loss of academic interest and/or an attitudinal problem ³². It is argued that this negative self-perception sets in motion a perpetual cycle of negative evaluation that may persist in spite of evidence to the contrary ⁶⁶. This affect takes on a greater significance after hearing many students’ common frustration at not having the ability to communicate verbally despite many years spent learning English. Such an environment affects students’ perception of learning and understandably de-motivates students which is reflected in regular polls of respondents who acknowledge that while there is growing need for English many indicate they had negative feelings towards the language ³². Without a sense of academic achievement fails to encourage a sense of accomplishment, a sense of value in the instruction itself, and a resultant confidence boost. Therefore, although the aim of active learning is not to perfect students’ production of the target language, this is how it will be viewed by many Japanese students, resulting in any failure to achieve this being perceived negatively. The perception has led the author on many occasions to explain to students about the aim of the active learning’s tasks and the pedagogic benefits of engaging in the task activities. Despite this reassurance students failed to see any real achievement, resulting in for example, a lack of participation with their group members.

6. Conclusion

Credible evidence in support of active learning in promoting academic achievement and positive student attitudes is compelling and should stimulate faculty to transition to such nontraditional models of teaching and learning. Similarly, support for collaborative and cooperative learning challenges traditional assumptions that individual work and competition best promote achievement ⁶⁷. Although active learning appears to have strong theoretical and pedagogical arguments, basing an entire lesson around this activity may lead to less productive lessons due to Japanese students’ focus on ‘completing’ the task rather than as a means of improving communicative competence. Omission of a more focus on form seems to restrict the goal to ‘teaching how to do tasks better’ ¹⁰ rather than providing opportunities for students to focus not only on language but also on the learning process itself. The type of interaction produced therefore raises questions about how much students’ language proficiency is being developed. It is clear from the findings form the author’s communicative class that tasks (in the case of task-based leaning) could over emphasize the importance of just ‘getting the job done’ at the expense of improving target language ability ². This is supported by researchers ⁶⁸ who highlight that task-based learning’s interaction often seems unimpressive as there is:

A tendency to produce very indexical interaction, i e. interaction that is content-bound, inexplicit, and hence obscure to anybody reading the extracts. Utterances at the lowest level of explicitness necessary to the successful completion of task ⁶⁸.

Without a more structured element, assuming that ‘awareness’ will suddenly occur is also presumptuous as it fails to provide the correct context for Japanese students. The desire for students to be able to negotiate real-life situations is a learning objective most English language teachers would aspire to, however, a teaching approach which places too heavy a burden on students is not only unrealistic but also unreasonable. Few would argue that the teacher dominated, initiation–response-feedback pattern needs to be used more often, but the other extreme of merely ‘furnishing conditions in the classroom’ ¹⁴ also seems equally undesirable. An over-emphasis on techniques rather than outcomes renders active learning bound to means and not to ends. Due to the setting and the cultural factors highlighted above many of these cannot be overcome to any significant degree, regardless of the teaching methodology. Their influence may be minimized but this would require teachers acknowledging and bearing the responsibility to adopt teaching methods/methodology which recognise their influence. To some degree teachers must also evaluate their expectations or goals which may mean adopting activities which seem too teacher-centred yet meet student expectations and maximise involvement in the learning process.

The opportunity to ‘use the language freely’ can be appreciated by most foreign faculty in Japan yet it presupposes a certain level of linguistic competence. There is a lot of practice that is required before this can be realistically achieved, from simple, restricted practice exercises to more complex, elaborate and lengthy activities. A learning process that realizes not only this process but also the effect on the student, would appear to more beneficial. Therefore activities must be well designed cognitive activities that promote thoughtful engagement around learning outcomes ¹³. In designing sequences of tasks it is important to consider that this process is recognized in the salience of the pedagogic goals of the task ¹⁰ with particular emphasis placed on activities which aim to provide students with a sense of achievement and personal accountability in their own learning and encouraging them to think about the process of language learning and how to approach it more effectively. There is nothing inherent in typical active learning activities that suggest they alone provide thoughtful engagement (certain ‘passive’ learning activities may also promote thoughtful engagement when structured appropriately). Also, focus on the teaching of the principles of interaction, interdependence, and individualism in the language learning process should also be explicitly laid out. These ‘cooperative strategies’ may alleviate the otherwise negative self-perceptions that evolve from poor individual performances ⁶⁹. Repeated failure would be de-motivating for students so the concept of a ‘reasonable challenge’ has to be realistic in what can be achieved.

Biography

Christian Burrows has been teaching at tertiary education in Japan for more than twenty years. Since April 2021 he has been teaching at the Hiroshima Prefectural University, Japan. He is currently conducting research into communication strategies and how they can be used to overcome communication barriers to improve speaking proficiency.

Notes

{1}. The English level of the majority of students was CEFR B1 at the beginning of the semester.

{2}. Eiken is an abbreviation of the Japanese Jitsuyo Eigo Gino Kentei (Test in Practical English Proficiency), one of the most widely used English-language testing programs in Japan. A pass score on Eiken Grade 3 (1,456 points) equates to CEFR A1 level. A score over 1,699 points (out of 2,220) represents CEFR A2 level. A perfect score represents B1 level.

{3}. A pass score on Eiken Grade 2 (1,980 points) equates to CEFR B1 level. A score over 2,299 points (out of 2,600) represents CEFR B2 level. A perfect score represents the border between B2 and C1 level. For Grade Pre-2, a pass score (1,728 points) equates to CEFR A2 level. A score over 1,949 points (out of 2,400) represents CEFR B1 level. A perfect score represents B2 level.

{4}. After leaving college approximately 55% of high school students continue on to university.

{5}. NB: The TOEIC test is a commonly used English proficiency test that is designed to assess a non-native English-speaker’s ability to use English successfully in the workplace. In 2018, over 7 million TOEIC tests were administered in approximately 150 countries, by more than 14,000 companies and other organizations. Total numbers for 2018 show that the overwhelming majority of test takers (70%) were in Japan and Korea.

{6}. The review has also been able to search the Japanese language literature systematically, although a number of databases in Japanese were searched no non-English language reports were retrieved. In all but one case abstracts have been available for the Japanese studies identified.

References