Journal Search Engine

ISSN : 1229-3431(Print)
ISSN : 2287-3341(Online)

Journal of the Korean Society of Marine Environment and Safety Vol.24 No.3 pp.311-318
DOI : https://doi.org/10.7837/kosomes.2018.24.3.311

A Study on the Effectiveness and Improvement of Simulation Training for Apprentice Officers

Myoung-ki Lee^*

, Young-Soo Park^**†, Weon-Jae Ha^**

^*Ocean Science and Technology school, Korea Maritime and Ocean University, Busan 49112, Korea
^**College of Maritime Sciences, Korea Maritime and Ocean University, Busan 49112, Korea

Corresponding Author : youngsoo@kmou.ac.kr, 051-410-5085

Received 20180312 Review 20180424 Accepted 20180529

Abstract

In accordance with the stipulations of the STCW Convention, simulation training has been enforced in order to develop practical skills so as to prevent accidents by predetermining the risks in special marine environments. Simulation training is a useful way to acquire navigation abilities, and can continuously measure the ability of a trainee by applying an appropriate evaluation. However, the result of training is evaluated by the instructor’s subjective judgment without quantitative criteria. Therefore, this study aims to quantitatively evaluate the effectiveness of simulation training. For this purpose, evaluation items were derived by analyzing legal standards, earlier studies, and the current status of MET institutions. The simulations were then performed three times in the same scenarios and analyzed the results. As a result, it has been shown that the objectively analyzed ability to keep the route and to make safe passage with other vessel, as well as subjectively evaluated ability by the apprentice officer has been improved as training progressed. Through the evaluation of simulation training results, it can be derived that simulation education needs supplementation, and can be provided as a basic form of data to quantify the evaluation results of the simulation training in the future.

Key Words : Effectiveness of simulation training , Subjective evaluation , Cross Track Distance , Clearance distance , Learning curve

초록

키워드 :

This article has been cited by 0 article in crossref

Cited-By

Funding:

1 Introduction

A navigator needs to possess practical skills of navigation with the good use of navigational equipment, as well as overall knowledge of navigation. In particular, they should quickly and accurately make a decision in dangerous situations. Therefore, in accordance with the International Convention on Standards of Training Certification and Watchkeeping for Seafarers (STCW) II/1 and IMO Model Course 1.07, completion of Automatic Radar Plotting Aid (ARPA) training and radar simulation training are required (IMO, 2010). Domestically, in accordance with The Ship Personnel Act (http://www.law.go.kr, 2017), ship officers should have radar simulation training based on a ship handling simulator (5 days) as well as Automatic Radar Plotting Aid training (3 days) in order to have the necessary practical skills for predetermining the risks inherent in special marine environments, and preventing accidents.

Simulation training can be repeated in a safe and controllable situation, and enables instructors to provide proper feedback and evaluate the trainees in a useful learning method so they may acquire the practical skills of navigation (Nahlinder, 2010). By applying proper evaluation methods, it is possible to measure a trainee's ability continuously during simulation training, and observe the process of a trainee's competency improvement (Kobayashi, 2005).

Regarding the simulation training specifically, Karlsson (2011) suggested the items to be evaluated by an instructor at the time of briefing and debriefing, and Park (2016) classified and suggested the items to be evaluated depending on the basis of different roles during simulation training. However, the suggested items, including proper CPA maintenance and a CPA/TCPA setting, were mostly based on an instructor's subjective judgment. Shin et al. (2017) conducted a questionnaire survey on the improvement of equipment utilization ability, radar analysis ability, maneuvering ability and comprehensive ability and performed control analysis and proximity analysis of the simulation result. However, in the study, their improvement was evaluated by trainers' subjective judgment, and the analysis of the simulation result failed to take into account passage with other vessels.

Therefore, this study aims to analyze the effectiveness of simulation training in an objective way. For this purpose, the main items that help to evaluate the training effectiveness were drawn by analyzing legal standards on simulation training, previous studies, and the current simulation training evaluation status of MET (Maritime Education Training) institutions. While performing a 64-hour simulation training program, the result of each simulation after 4 hours, 24 hours, and 48 hours was evaluated and compared by derived evaluation items.

Through the evaluation of the simulation training results, it is possible to draw out the complementary point and improvement plan for simulation training, and to provide a basic data set in order to quantify the evaluation results of the simulation training.

2 Extraction of main items for simulation training

2.1 Survey on legal standards for simulation training

In order to find the evaluation criteria defined to prove one's navigational competency related to Radar/ARPA simulation training, legal standards for simulation training were surveyed.

Table 1 presents a comparison between international and Korean legal standards for simulation training.

Regarding international standards, STCW (International Convention on Standards of Training, Certification and Watchkeeping for Seafarers) Code includes navigational competency items as the minimum compulsory standards of ship officers in charge of navigation in a ship with more than 500 tons. In Part A / A-II / 1 of the Code, the items related to the use of Radar and Automatic Radar Plotting Aids (ARPA) for keeping safe navigation are described. Although the evaluation items related to information acquisition and analysis based on Radar/ARPA, actions to avoid collision, changes in a ship's course and speed, communication, and maneuvering signals are described, the evaluation criteria of the items are not defined. IMO Model Course 1.07 suggests the guidelines for training program on the basis of the minimum standards of STCW, and the necessity of evaluation is described, but evaluation items or criteria are not defined (IMO, 2017).

With regard to Korean legal standards, Radar/ARPA training contents and training hours are defined to prove an officer's qualifications in accordance with Article 2 of the Enforcement rule of the Ship Personnel Act. The curriculum of such training is defined in Article 14 of the Notice of Standard for Designation Education Institution by the Ministry of Oceans and Fisheries. However, items about training evaluation are not defined in the Ship Personnel Act and Notice of Standard for Designation Education Institution.

2.2 Survey of earlier studies on simulation training

To draw the main items for the evaluation of effectiveness of simulation training, the evaluation items of simulation training suggested or used in previous studies were surveyed.

Table 2 presents a comparison of evaluation items suggested or used in previous studies.

To evaluate trainees, Karlsson (2011) suggested evaluation items in the categories of simulation and debriefing. These items include CPA/TCPA limit setting and minimum 1 mile CPA maintenance. In his study, whether or not items were executed was evaluated.

Park (2016) classified roles into captain, officer, VTSO, and Q/M in simulation, and suggested different evaluation items depending on the roles. The items include proper CPA maintenance and proper radar use. The evaluation score ranged from 1 to 10 points.

Shin et al. (2017) conducted a subjective evaluation and simulation analysis. In the subjective evaluation, items such as Radar/ARPA utilization ability, radar analysis ability, and maneuvering ability were evaluated in the points scale ranging from 1 to 10. In simulation analysis, control and proximity were analyzed.

Given the analysis results of previous studies, it was found that Karlsson (2011) and Park (2016) commonly suggested RADAR use and maintaining proper CPA for safe passage as items for evaluation, and that Shin et al. (2017) evaluated simulation training through trainees' subjective evaluation and simulation based proximity and control analysis.

2.3 Survey of the status of MET institutions for simulation training

To draw the main evaluation items in order to analyze the effectiveness of simulation training, the evaluation items which are used in Korean MET institutions were analyzed in the categories of communication, safe passage with other vessels, implementation of the voyage plan, and usage of equipment.

Table 3 shows a comparison of simulation training evaluation items between Korean MET institutions.

In institution A, there are 6 evaluation items pertaining to communication, 3 items focused on safe passage with other vessels, 5 items on the implementation of the voyage plan, and 2 items about the usage of equipment. Basic points are given, and a final score is calculated by way of adjusting points on the basis of evaluation criteria.

In institution B, there are a total of 14 items: no evaluation item for communication, 7 items pertaining to safe passage with other vessels, 1 item on the implementation of the voyage plan, and 6 items on equipment usage.

In institution C, there are 2 evaluation items pertaining to communication, 7 items on safe passage with other vessels, and 1 item about the usage of equipment. The overall score is calculated depending on whether each item is executed properly.

The simulation evaluation items of each training institution were surveyed in the categories of communication, safe passage with other vessels, implementation of the voyage plan, and usage of equipment. It was confirmed that MET institutions mainly used items related to safe passage with other vessels.

In the items pertaining to safe passage with other vessels, collision avoidance action and CPA/TCPA, were commonly found; in the items about communication, reporting and communication were commonly included; in the items on implementation of the voyage plan, course check were commonly found; in the items about the usage of equipment, use of radar was frequently found.

2.4 Extraction of evaluation items for simulation training

To evaluate the effectiveness of simulation training, legal standards of simulation training, earlier studies, and the evaluation items of Korean MET institutions were surveyed. As a result, three evaluation items apprentice officers' subjective evaluation, cross track distance, and clearance distance with other vessels were each derived.

(1) In terms of apprentice officers' subjective evaluation, their evaluation items on navigation and overall skills which were used in Shin et al. (2017) were scored in the categories of ship-handling ability and navigation ability. Fig. 1 shows an example of apprentice officers' subjective evaluation.
(2) Cross track distance is the item that evaluates the ability to keep course line. Course keeping, as suggested by Park (2016), and position fixing, which is commonly used in Korean MET institutions, are both used to evaluate the ability to keep course line so as to quantify the cross track distance from a planned route. Fig. 2 illustrates the outline of cross track distance.(1)(2)

C r o s s T r a c k D i s \tan c e = sin θ \cdot l

(1)

θ = C O S^{- 1} (\frac{b^{2} + l^{2} - a^{2}}{2 b l})

(2)

a : The distance from the previous way point to the current position (m)
b : The distance from the previous way point to the next way point (m)
l : The distance from the current position to the next way point (m)
θ : The angle between b and l (°)

(3) In regards to the clearance distance with other vessels, the item of safe passage which is reflected via the item related to CPA which was suggested by both Karlsson (2011) and Park (2016) and is commonly used in Korean MET institutions was used to quantify the clearance distance with other vessels.(3)

C l e a r a n c e D i s t a n c e = \sqrt{{(x_{o} - x_{t})}^{2} + {(y_{o} - y_{t})}^{2}}

(3)

(x_o, y_o) : The coordinates of own ship's position
(x_t, y_t) : The coordinates of a target ship's position

3 Analysis on the result of simulation training

3.1 Experimental subjects and scenario setting

A 64-hour simulation training was conducted for 85 apprentice officers who had completed one year of onboard training. They were subjected to the same-scenario simulation after 4, 24 and 48 hours of training, and the results of the simulation were analyzed. The waterways in the scenario ware set to the Kanmon Straits, the most difficult one as reported by apprentice officers in simulation training (Shin et al., 2017). Kanmon Straits AIS data between 0800-0900 hours were used because it has the largest actual East-bound traffic volume in the Straits (Hiroaki et al., 2010). As a target ship type, a pure car carrier with good ship-handling performance was selected among vessels with less than 10m of draft which can pass the Kanmon Straits (UK Hydrographic Office, 2013).

Fig. 3 illustrates the waterways and the ship used in the scenario designed for the simulation training evaluation.

3.2 Analysis on the effect of each evaluation item

In order to analyze the effectiveness of simulation training, the results made after 4-hour, 24-hour, and 48-hour simulation training were analyzed in terms of the apprentice officers' subjective evaluation (ship-handling ability, navigation ability), cross track distance, and clearance distance with other vessels.

(1) Subjective evaluation of apprentice officers

At the end of each simulation, apprentice officers' subjective evaluation was performed to find their ship-handling ability and navigation ability. These apprentice officers played the role of a captain or an officer. Ship-handling ability refers to their ability to change course in a proper direction, and the navigation ability represents their ability of overall navigation skills in simulation. Fig. 4 shows graphs of the average self-assessed points of apprentice officers' derived from their subjective evaluations on their ship-handling ability and navigation ability.

As shown in Fig. 4, as the simulation training progresses, both ship-handling ability and navigation ability subjectively evaluated by apprentice officers gradually improved. In order to find out whether these changes were significant, a paired sample t-test was conducted. The result of this is presented in Table 4.

A paired sample t-test was also conducted for each simulation. As a result, ship-handling ability and navigation ability were shown to significantly differ between simulation 1 (after 4-hour training) and simulation 2 (after 24-hour training) with a p-value of .000. The mean increased between simulation 2 (after 24-hour training) and simulation 3 (after 48-hour training), but it was not significantly different due to having a p-value of more than 0.05. Given the analysis result, it was concluded that apprentice officer would subjectively feel improvements in their abilities after 52-hours of simulation training, and in particular that they would subjectively feel great improvements at the beginning of training.

(2) Cross track distance

The Kanmon Straits features a narrow width and a large passage amount so that there is a high risk of accidents. Therefore, leading lights are used to induce a vessel into a safe route. In this simulation, a vessel was supposed to steer a course of 141 degrees along the leading light Oseto No.1. To evaluate the apprentice officers' ability to keep course line, cross track distance was measured. For a deviation to the starboard, a ' ' sign was used; for a deviation to the port, a '+' sign was used. Fig. 5 illustrates the graphs of the average cross track distance for each simulation.

As shown in Fig. 5, as simulation training progressed, the average cross track distance reduced to 224.23 m, 197.36 m, and 90.09 m, corresponding to the order of the simulations. In particular, the difference between simulation 1 (after 4-hour training) and simulation 3 (after 48-hour training) was 134.14 m. Therefore, their ability to keep course line was greatly improved.

Table 5 shows the results of paired sample t-test which was conducted to find out if there was any significant differentiation in the average cross track distance between each simulation.

Significant variation was observed between simulation 1 (after 4-hour training) and simulation 2 (after 24-hour training), and between simulation 2 (after 24-hour training) and simulation 3 (after 48-hour training) with a p-value of .000. The standard deviations were also reduced. Therefore, as training progressed apprentice officers who had deviated from a route were seen to keep a route within a certain range.

(3) Clearance distance with other vessels

In order to evaluate safe passage with other vessels, the average clearance distance between an officer’s own vessel and other vessels was measured. In this case, to exclude distant vessels, only vessels passing within 1,200 m were surveyed. This value is calculated by setting the maximum distance of the Kanmon Strait to about 1000 m and the clearance to about 200 m.

Fig. 6 illustrates the average clearance distance with other vessels in each simulation.

The average clearance distance with other vessels was 792.15 m, 848.82 m, and 854.60 m, respectively. As simulation training progressed, navigation was made with a wider distance from other vessels. In particular, the clearance distance with other vessels between simulation 1 (after 4-hour training) and simulation 2 (after 24-hour training) increased by 56.67 m, and the clearance distance with other vessels between simulation 2 (after 24-hour training) and simulation 3 (after 48-hour training) slightly increased by 5.78 m. To find out if these changes were significant, a paired sample t-test was conducted on each simulation. The result is presented in Table 6.

This revealed a significant difference between simulation 1 (after 4-hour training) and simulation 2 (after 24-hour training) with a p-value of .000. No significant difference was found between simulation 2 (after 24-hour training) and simulation 3 (after 48-hour training) with a p-value of .877. In simulation 1 (after 4-hour training) and simulation 3 (after 48-hour training) with a p-value of .000, as simulation training progressed, apprentice officers' ability of safe passage with other vessels was improved. In the t-test, the ability had greatly been improved following the first 24 hours of training.

3.3 Comprehensive evaluation

To find the effectiveness of the simulation training, the apprentice officers' subjective evaluation, cross track distance, and clearance distance with other vessels were all analyzed. Fig. 7 comprehensively presents the results from the analysis on each evaluation item in graph form.

As shown in Fig. 7, the effect of training during a certain period of time was large in all areas except cross track distance, but the effect diminished after a certain period of time. This can be explained in comparison with the learning curve of pedagogy. This learning curve is a graphical representation of one's actions that change as a result of practice. At the beginning of learning, this curve is changed to a large degree. After a certain period, the learning curve remains unchanged in a sort of standstill state. Still later, the curve is also highly changed from this ‘recovery period’ (Education Research Institute Seoul National University, 2011).

To shorten the standstill state period, it is necessary to change a scenario or a training method after 24 hours. In case of cross track distance, the effect was small at the beginning of the training. This is because the apprentice officers' lack of steering ability and navigation ability lead to excessive collision avoidance action. Therefore, it is necessary to emphasize the importance of course-keeping during the beginning of training.

4 Conclusion

In accordance with legal standards, simulation training has been enforced in order to develop practical skills for the prevention of accidents by predetermining the risks present in a special marine environment. This study attempts to evaluate the effectiveness of simulation training, and to draw something to improve the training. The results are presented as follows:

(1) In order to draw the main items for evaluation of the effectiveness of simulation training, legal standards of simulation training, previous studies, and the current status of MET institutions were surveyed. As a result, CPA/TCPA related items for safe passage with other vessels, and the item of position fixing, were found to be the primary evaluation items.
(2) In order to evaluate the effectiveness of simulation training, simulations were performed in the same scenario after 4-hour training, 24-hour training, and 48-hour training periods in the 64-hour simulation training program. Cross track distance, clearance distance with other vessels, and apprentice officers' subjective evaluation (ship-handling ability, navigation ability) were used as evaluation items. As a result, it was found that not only were apprentice officers' ship-handling ability and navigation abilities improved, but their ability for course-keeping and their ability of safe passage with other vessels were improved as well.
(3) Given the comprehensive analysis on the effectiveness of simulation training, the learning effect was still in standstill state after 24-hours of training. In order to shorten the state period and maximize the effect of training, it is necessary to change a scenario or training method. Given that the ability to keep course line was less improved than other abilities at the beginning of training, it is necessary to emphasize the importance placed on course-keeping.

Based on one scenario, this study used officers' subjective evaluation, cross track distance, and clearance distance with other vessels in order to evaluate the effectiveness of simulation training. In the future, more scenarios with various waterways and ship types, and more evaluation items including measurements of vessel speed and position fixing will be incorporated in order to come up with the criteria necessary to verify the objective effect of simulation training.

Figure

Fig. 1..

Apprentice officer’s self evaluation score sheet.

Fig. 2..

Cross Track Distance.

Fig. 3..

Waterways and ship type.

Fig. 4..

Average comparison of navigator’s evaluation.

Fig. 5..

Average comparison of cross track distance.

Fig. 6..

Average comparison of clearance distance.

Fig. 7..

Comprehensive evaluation result.

Table

Table 1.

Comparison of legal standards for simulation training

	Evaluation standard
International standards	STCW	· Information obtained from Radar/ARPA is correctly interpreted and analyzed · Action taken to avoid a close encounter or collision with other vessels is in accordance with COLREG · Decisions and Adjustments made to the ship’s course and speed maintain safety of navigation · Communication is clear · Maneuvering signals are made at the appropriate time and are in accordance with COLREG
IMO Model Course	Only the need for evaluation is mentioned
Korean standards	Ship Personnel Act	Not mentioned
Standards for MET	Not mentioned

Table 2.

Comparison of evaluation items of earlier studies

Author	Evaluation items
KARL SSON (2011)	During Exercise	Set up and maintain Radar display, Set CPA and TCPA limits, Manually acquire targets, Display true vectors, Vary vector length (Vary Radar range), ARPA display in True, North Up, Relative Motion & Head Up, Display ground stabilized (bottom track), Display sea stabilized (water track), Obtaining range & bearing to targets, Trial maneuver with both course and speed change, Activate/Silence the lost target alarm, Cancel a single target, Any violation of COLREGS, Set up and auto acquisition zone
Debriefing	Understanding the differences between relative and true vectors, Motivating his/her actions during the exercise, Any violation of COLREGS, Minimum CPA>1.0mile, Using TRIAL Manoeuvre, Understanding the difference of using speed over ground or through water, Using all resources available
PARK (2016)	Captain	Proper order, Effective Bridge Resource Management, Proper CPA maintain, Safety passage, Cooperation with other ships, Proper Speed Maintain, Proper VHF Communication
Officer	Understanding Course Line, Proper Positioning, Providing Proper Information to Captain, Identifying Dangerous ships, Proper Radar Usage, Proper VHF Communication
VTSO	Providing Proper Information to Ship, Management of Proper Traffic Flow with other ships, Proper VHF Communication
Q/M	Repeating for Rudder Order, Completing Report to Officer (Captain), Proper Course Keeping, Fatality of Rudder Usage, Manual Steering
SHIN et al. (2017)	Selfevaluation	Radar ARPA utilization, Radar decoding skills, Ship handling and overall skill
Simulation Analysis	Control analysis, Proximity analysis

Table 3.

Comparison of evaluation for MET institution

MET Institution	Evaluation items
A	Communication	Proper Order, Proper VHF Communication, Providing proper information, Using SMCP, Repeating of rudder order, Completing report to officer
Safe passage with other vessel	Accident (Collision) Occurrence, Proper CPA maintenance, Identifying Dangerous ships
Implementation of the voyage plan	Accident (Grounding) Occurrence, Proper speed maintenance, Understanding course line, Proper positioning, Course keeping
Usage of equipment	Proper radar usage, Familiarization with navigation equipment
B	Communication	-
Safe passage with other vessel	Action to avoid collision using ARPA, Taking proper action to avoid collision in a timely manner, Large alternation of course, Alternation of course direction, Compliance with COLREG, Look-out, Plotting by risk judgment
Implementation of the voyage plan	Position fix using Radar
Usage of equipment	Radar operation according to procedure, Radar/ARPA setting, Using Parallel Index Line, Resetting the Radar according to weather changes, Use EBL / VRM, Familiarization with Radar/ARPA
C	Communication	Simple and clear communication, Reporting in a clear manner
Safe passage with other vessel	Report bearing of target, Report distance of target, Report relative movement of target, Report DCPA, Report TCPA, Report the risk of collision, Suggestion of action to avoid collision
Implementation of the voyage plan	-
Usage of equipment	Use of plotting function

Table 4.

T-test result of apprentice officer’s evaluation

	Mean	SD	t	p-value
Ship Handling Ability	1st simulation	7.33	1.96	-3.602	.001
2nd simulation	8.56	1.32
2nd simulation	8.56	1.32	-1.158	.252
3rd simulation	8.83	1.34
1st simulation	7.33	1.96	-4.422	.000
3rd simulation	8.83	1.34
Navigation Ability	1st simulation	7.47	1.72	-3.630	.001
2nd simulation	8.58	1.27
2nd simulation	8.58	1.27	-.468	.642
3rd simulation	8.68	1.33
1st simulation	7.47	1.72	-4.005	.000
3rd simulation	8.68	1.33

Table 5.

T-test result of cross track distance

	Mean	SD	t	p-value
1	1st simulation	224.23	188.53	4.267	.000
2nd simulation	197.36	161.80
2	2nd simulation	197.36	161.80	24.517	.000
3rd simulation	90.09	61.32
3	1st simulation	224.23	188.53	27.625	.000
3rd simulation	90.09	61.32

Table 6.

T-test result of clearance distance

	Mean	SD	t	p-value
1	1st simulation	792.15	270.15	-8.214	.000
2nd simulation	848.82	261.10
2	2nd simulation	848.82	261.10	.155	.877
3rd simulation	854.60	240.39
3	1st simulation	792.15	270.15	-10.692	.000
3rd simulation	854.60	240.39

Reference

Education Research Institute Seoul National University (2011) Glossary of education terms, http://terms.naver.com/list.nhn?cid=42126&categoryId=42126
S. Hiroaki , O. Mirai , U. Hideo , S. Masatoshi (2010) Marine Traffic Analysis of Three Major Bays Ship Handling Difficulty of Ship Equipped with AIS., Japan Institute of Navigation, Vol.Vol. 123 ; pp.13-19
IMO (2010) Manila Amendments to the Seafarer’s Training, Certification and Watchkeeping (STCW) Code 2010 Manila Amendments,, http://krcon.krs.co.kr/
IMO (2017) IMO Model Course 1.07 Radar Navigation at Operational Level.,
T. Karlsson (2011) The importance of structured briefings & debriefings for objective evaluation of ARPA simulator training, Ph.D. Goteborg: Chalmers university, pp. 31-41.,
H. Kobayashi (2005) Use of Simulators in Assessment, Learning, and Teaching of Mariners, WMU Journal of maritime affairs,, Vol.4 (1) ; pp.57-75
Korea Ministry of Government Legislation (2017) Ship Personnel Act, http://www.law.go.kr
Korea Ministry of Government Legislation (2017) Enforcement Decree of the Ship Personnel Act, http://www.law.go.kr
S. Nahlinder (2010) Flight Simulator Training : Assessing the potential. Linkoping: Linkoping studies in Science and Technology, Dissertation No. 1250.,
Y.S. Park (2016) A Study on the Standardization of Education Modules for ARPA/Radar Simulation., Journal of The Korean Society of Marine Environment & Safety, Vol.22 (6) ; pp.631-638
D.W. Shin , Y.S. Park , D.H. Kim (2017) A Study on the effect of ARPA/Radar Simulation Training., Korean Society of Marine Environment & Safety, Vol.23 (3) ; pp.294-300
UK Hydrographic office (2013) ADMIRALTY Sailing Directions Japan Pilot., UKHO, Vol.Vol. 3 ; pp.56