1 | P a g e  

 

National Institute of Open Schooling Academic Department

Adolescence Education Programme

National Institute of Open Schooling (NIOS), India

United Nations Population Fund (UNFPA), India

Report

“Review and Updating of the Monitoring Protocols” (24th June – 26th June 2015)

A three day workshop “Review and Updating of the Monitoring Protocols” was held from June 24- 26, 2015 at NIOS, Conference Room, 4th Floor, A-24/25, Sector 62, NOIDA. The workshop was organized under the UNFPA_MHRD supported Adolescence Education Programme. The workshop was organized to review the Monitoring tools with the SAIED lens. Participants of the workshop were drawn from different SAIED institutions. OBJECTIVES

With an aim of qualitative up-gradation of the monitoring mechanism by making it more focused the objective is to:

review the monitoring tools in the Academic Facilitator’s Handbook with respect to the changed administrative and academic functions;

articulate the changed administrative and academic functions in the norms and procedures in the monitoring tools in the handbook;

develop items eliciting feedback regarding quality of academic work during PCPs and utilization of infrastructure in AIs, AVIs, AAs, SAIED.

METHODOLOGY

Step 1: As a first step the monitoring tools were revised by the SSS Department. These were then forwarded to the Regional Centers for field testing and feedback.

Step 2: Based on the observations received from the Regional Centers, the Monitoring Tools were redrafted by AEP Unit.

Step 3: The redrafted tools shall be reviewed and finalized by selected members drawn from various stakeholders. The members shall be nominated by the competent authorities.

Step 4: The finalized Monitoring Tools thus obtained shall be presented before DAB and AC for final approval.

TIME LINE:

2 | P a g e  

 

Step 1 Jan-March, 2015 Field testing at all possible locations

Step 2 April-May, 2015 Consolidation of Feedback, Redrafting of Tools

Step 3 June-July, 2015 Brainstorming and Ideating

Redesigning the tool in collaboration with Student Support Department based on the suggestions of the officials of Academic Dept, Vocational Dept, OBE Unit, Media Unit, Capacity Building Cell, Regional Centre along with selected Academic Facilitators and Coordinators.

DAY 1: JUNE - 24, 2015 INAUGURAL SESSION: Dr. Kuldeep Agarwal, Director Academic welcomed Mr. C. Dharuman, Director Student Support Services, Prof. Madhu Parhar STRIDE, IGNOU and all the participants. . The session began with a round of self introductions by all the participants. In his welcome address Dr. Agarwal stressed on the importance of Monitoring for success of any programme. With reference to NIOS he said that the role of Academic Facilitator (AF) in the process of monitoring and evaluation of the Accredited Institutions (AI’s) is very crucial. Director Academics also spoke about the importance of AE issues and its implementation and integration in study materials. He also informed the participants that the process of revision of tools had been taken up jointly by SSS Department and Academic Department and discussed that strengthening of monitoring tools is the core of this workshop. In his opening remarks, Mr. C.Dharuman, Director SSS apprised the participants about the various activities and initiatives taken up by the NIOS for monitoring and evaluation. He also spoke about the various challenges faced by the system like lack of support from the Academic Facilitators, unsatisfactory tracking of the conduct of PCPs and practical. He said that Department of Student

3 | P a g e  

 

Support Services was committed to bring about reforms in monitoring. He said that the “Handbook for Academic Facilitators” developed by NIOS would be suitably revised and put up through the process of final approval by the Academic Council. Thereafter, the Resource person Prof. Madhu Parhar, STRIDE, IGNOU spoke about the proper planning with a pre defined input for an appropriate output. She emphasized on the importance of not just monitoring but setting up of a proper evaluation system for tracking the progress and process for overall improvements in strengthening of the system. The Vote of Thanks was delivered by Ms.Asheema, Project Coordinator (AEP) Session 1: Activity 1- Ice Breaking Session Activity 2- Power point presentation ‘Workshop Objectives and Methodology’ Activity 3- Bringing participants on the same page about Monitoring Methodology: Brainstorming session based on four cartoons leading to common understanding regarding role of monitoring on programme implementation, clear understanding of goal and purpose of monitoring. Resource Person: Prof. Madhu Parhar Rapporteuring: Ms. Ankita Juneja

Ms. Asheema Singh After the ice breaking session, the workshop objectives and methodology was presented by Ms. Asheema Singh. She outlined the stepwise approach for reaching the outcome through group consensus. Participants were divided in four groups and a cartoon and charts were given to each group. They were asked to discuss the situations represented in the cartoons and present their views in light of Goals, Purpose and Objectives. Cartoon 1: The participants discussed that only relevant information should be asked. Some participants contradicted that sometimes irrelevant information may also help. Ms. Madhu Parhar concluded that Random information will not help and proper frame work is required. So, the information and data should be both Qualitative and Quantitative for which the right methodology should be selected and the objectives had to be clearly defined.

4 | P a g e  

 

Cartoon 2: The group discussed that the cartoon depicts a lot of confusion and mismanagement. Clearly the driver of the project is not clear regarding the goals and direction. Despite evaluations results in hand the whole team is heading towards a wrong path as they are unable to derive the befit out of the evaluation report. Ms. Singh concluded that if there is a coherent relationship between Goals, Objectives and Purpose then remaining on the right course is easier.

Cartoon 3: With regards to Cartoon 3, participants discussed that everybody has worked to get the attention required but are not clear of about what they have to do after that. Therefore, it is important to be clear about the framework and goals beforehand in order to maintain the attention and to avoid any failures. Prof. Parhar highlighted the importance of teamwork with the role of Regional Directors, Academic Facilitators, Tutors, Learners, Coordinators etc. under the direction of student support department.

Cartoon 4: While discussing Cartoon 4, participants said that if the goal is clear, the sailor will sail the boat in the right direction and depicted water as the ocean of knowledge and spoke about the role of reviewing process for monitoring. In her closing remarks Prof. Parhar flagged the issue of installation of evaluation framework. Ms. Singh cautioned that best tools may not help an organization until the information obtained is used for frequent programme assessment and course correction.

5 | P a g e  

 

Then all four groups were given VIPP cards. Group 1 was asked to discuss and list the “Goals”, Group 2 the “Objectives”, Group was expected to write the “Purpose” and Group 4 was asked to write “What not to do” during the process for developing tools of Monitoring and Evaluation. All the groups presented their lists. All the points were discussed in a brain storming session. The finalized list was reflected on chart papers as a visual cue for all the participants.

Keeping in view that the concerns of various departments was to be addressed in the same tool, cartoon 5, was used as a tool of caution.

6 | P a g e  

 

Session -2: Activity 1: Energizer- Eye-opener ‘Guess What’ Activity 2: A Power Point Presentation by Prof. Madhu Parhar on “Concepts of Monitoring Activity 3 – Overview of Monitoring and feedback requirements of:

Presentations by: Academic Department Vocational Department OBE Section Media Section ICT Section

Dr. Rajendra Nayak Ms. Radhika Dr. Rachna Bhatia Mr. Ravi Shankar Mr. S.K.Prasad

Resource Person: Prof Madhu Parhar

Ms. Asheema Singh  

Rapporteuring: Ms. Ankita Juneja

The participants were randomly divided into four small groups: Academic, Vocational, OBE and Media. Each group deliberated on what information to gather, the reason for gathering the information and how the gathered information would assist in bringing about a positive shift at both the policy and implementation level. Each group made a presentation of their work followed by Open house discussion with an objective of rationalizing and distributing the suggestions across the three monitoring tools. Post Lunch, participants played a game called “Eye Opener” which helped in understanding the core idea that the needs and issues of all stakeholders should be authentically articulated in the monitoring tools. Thereafter, Prof. Madhu Parhar explained the “Concepts of Monitoring” in which she defined Monitoring, Aims of Monitoring and types of indicators. She also discussed various models for Monitoring and Evaluation. She stressed upon the importance of Result based model for monitoring and evaluation and shared that this is the most commonly adopted model for a good outcome.

7 | P a g e  

 

The following then presented the specific monitoring needs of their respective departments –

Academic Department

Vocational Department

Media Unit

OBE Unit

ICT Unit The participants were then divided in to four groups i.e., Academic, Vocational, Media and OBE. Each group was given tool 1 for item analysis in context to their allocated department. Ms. Asheema Singh also explained different criteria on which the previous tools were revised. All group specific focus areas of monitoring were discussed in context with monitoring tools and were also given presentations from other departments for references

Prof. Madhu Parhar stressed on the objectivity of the information to make analysis and monitoring easy and appreciated the efforts made by NIOS. She also spoke about the implementation of a logical framework, target planning and result based evaluation to explore and identify any lacunae easily with improvisation of parameters and resource components.

8 | P a g e  

 

The debriefing session was conducted by Ms. Asheema Singh in which the participants participated actively. Following suggestions were given by different participants during the session-

Academic Facilitators needs to be more active and result oriented

Pre Performa for examinations may also be added to these tools

Conduction and implementation for PCP should be rigorously monitored.

Academic Facilitator could be used as a confidential observer for examination

Handbook should be common for all departments/ programmes

Number of visits and remuneration to the Academic Facilitator may be revisited.

The mechanism to deliver Certificates of learners may also be reviewed.

For effective monitoring of study centers Academic Facilitators need to be vigilant for all the process such as PCP sessions, examinations, practical and certification.

Trained retired teachers may be enlisted and oriented as an Academic Facilitator.

The sources of providing authentic information to learners may be increased.

More quality control and assurance methods need to be adopted

Use of resources for learners needs to be allocated, inspected and notified.

PTP Class schedules needs to be maintained for vocational department.

Labs and practical related material needs to be listed and maintained for learners

Accreditation should be discontinued if guidelines are not followed by study centers.

Media policies should be developed with an objective of adequate utilization of media with long term future planning.

The communication between AI’s and RC’s must be increased.

9 | P a g e  

 

DAY 2- JUNE 25, 2015

Session1:

Activity 1: Recap of the previous day activities.

Activity 2: Reviewing of pre-revised and revised monitoring tool for the first visit

Methodology: Open House Session

Resource Person: Ms. Asheema Singh Rapporteuring: Ms. Ankita Juneja

The second day of the workshop began with an insightful visit to media department of NIOS where all the participants explored the processes of their respective departments. Afterwards, Ms. Asheema gave a brief recap and summarization of the previous day’s activities with intent to focus on the goals, objective and purpose in order to revise the tools. The participants were divided into four groups Academic, Vocational, OBE and Media. Each group was given a copy of the pre-revised and revised monitoring tool for the first visit. The participants reviewed the tool against the role, goal and purpose of monitoring culled out in previous days sessions. The language of the tool and ease of transaction was also reviewed. The tool 1 was reviewed in context with the needs of Academic Department by Dr. Kiran Angra, Dr. Sandhya Kumar and Dr. Rajendra Nayak. The concerns of Vocational Department were looked into by Dr. Kirti Sharma, Mr. R.S.Rana and Ms.Radhika. Similarly the tool with reference to OBE Unit was reviewed by Mr. Subrata Chakravarty, Ms.Rachna Bhatia, Mr. P.M Pankaj and Mr. Narendra Singh. The concerns related to Media which included website were reviewed by the fourth group comprising of Mr. G.V.Shastri, Mr.S.K.Prasad, Mr.Ravi Shankar and Ms. Nina Gulani.

Session2:

Activity 1: Orientation on Scope of Statistical Analysis to check the reliability and validity of the first monitoring tool Activity 2: Analysis of Qualitative and Quantitative data in the light of Goals and purpose of monitoring

Resource Person: Ms. Asheema Singh Rapporteuring: Ms. Ankita Juneja

10 | P a g e  

 

Ms. Singh explained that all three tools include both Quantitative and Qualitative type of question. The objective is to review the qualitative questions and redesign them in order to reduce subjectivity. For the ease of understanding, Participants were given Tarang Tool for reference. The groups were asked to analyze Quantitative and Qualitative data that may be collected as interview, descriptions, anecdotes, opinions, interpretations, etc., in alignment with the goals, objectives and purpose of monitoring. The groups thoroughly examined each question of monitoring tool under the purview of statistical analysis for the formation of statistically valid and reliable tool while keeping in mind their group specific areas.

Session 3:

Activity 1: Energizer

Activity2: Group presentation and Open house discussion with an objective of rationalizing and distributing the suggestions on the First Monitoring Tool

Activity 3: Reviewing of pre-revised and revised monitoring tool for the second visit

Methodology: Open House Session

Resource Person: Ms. Asheema Singh Rapporteuring: Ms. Ankita Juneja

In post lunch session, Participants played a game that brought home the idea that it is important to hold all the objectives together while keeping the goal in mind.

11 | P a g e  

 

All the groups presented the First monitoring tool and gave their suggestions regarding the changes that should be made as per the group specific areas in the light of Goal, Purpose and objectives. All groups participated and contributed actively. The final changes made by each group were noted.

Afterwards, each group was given a copy of the pre-revised and revised monitoring tool for the second visit. The groups thoroughly examined each question of monitoring tool under the purview of statistical analysis for the development of statistically valid and reliable tool keeping in mind their group specific areas in alignment with the goals, objectives and purpose of monitoring. Hence, all group members actively gave their opinions to bring necessary changes required for interventions in the process of monitoring and evaluation.

The debriefing session was conducted by Ms. Asheema Singh. Following suggestions were given by different participants during the session-

Assessment of the study center may be done by Academic Facilitators and space may be provided in each Monitoring tool.

All the questions in the interview schedule may be sorted and rearranged on the basis of SLM, TMA e.t.c. as they are not arranged on their specific topics and headings mentioned.

These questions may be converted into close ended questions.

Visits during practical session should also be taken into account.

It was also proposed that the Academic Facilitators may check on availability of resources as mentioned in MOU, at the study centers.

It would also be valuable to understand the kind of materials, books etc the learners usually adopt for studying.

12 | P a g e  

 

DAY 3- JUNE 26, 2015

Session1:

Activity 1: Group presentation and Open house discussion with an objective of rationalizing and distributing the suggestions on the Second Monitoring Tool

Activity 2: Reviewing of pre-revised and revised monitoring tool for the Third visit

Methodology: Open House Session

Resource Person: Ms. Asheema Singh Rapporteuring: Ms. Ankita Juneja

The final day of the workshop began with groups presentations of the second monitoring tool. All group members demonstrated a high spirit of team effort and worked excellently in their allotted teams. The participants actively gave their valuable opinions and input. All the groups analyzed and discussed on different items given in the tool. Suggestions during open house discussions regarding reframing of items were recorded. The final changes made by each group were noted for finalizing of the second monitoring tool.

Each group was then given a copy of the pre-revised and revised monitoring tool for the third visit. Participants were expected to review the third monitoring tool against the role, goal and purpose of monitoring as explained in previous sessions.

Session 2:

Activity 1: Orientation on Scope of Statistical Analysis to check the reliability and validity of the Third monitoring tool

Activity 2: Analysis of Qualitative and Quantitative data in the light of Goals and purpose of monitoring

Activity 3: Group presentation and Open house discussion with an objective of rationalizing and distributing the suggestions on the Third Monitoring Tool

Resource Person: Ms. Asheema Singh Rapporteuring: Ms. Ankita Juneja

13 | P a g e  

 

The Third Monitoring tool was examined during this session. All the group members actively gave their opinions to bring necessary changes in all the three tools so that they should match the logical sequencing of processes.

Each group began with their groups presentations after reviewing and analyzing the third monitoring tool. The final reframing of questions was made by all the respective groups keeping in view the objectivity and placement as per their allotted areas. The Academic section and OBE section advocated evidences of documentation and use of facilities as per the guidelines as principal areas of monitoring. Media section focused on the availability and usage of ICT/media resources for learners while Vocational section reframed the tools involving the checking of implementation of Vocational Courses, availability of Vocational resource components, PTP scheduling and usage of laboratories and trade specific hands on facilities by the learners.

Valedictory Session:

During the valedictory session, participants from different regional centers gave their feedback to Mr. Dharuman Director SSS, and Dr. Kuldeep Aggarwal, Director Academics and expressed their gratitude for organizing this kind of innovative workshop. Talking about the importance of the workshop Dr. Kiran Angra, Principal Women’s Degree College Ambala, from Chandigarh Regional Centre said comprehensive awareness impacts the implementation of a programme. Hence, this was a timely action towards this direction. Mr. G.V.Shastri, Retd Principal. KVS from Hyderabad Regional Centre expressed that the workshop had enlightened him in all areas of NIOS and appreciated the fact that all experts utilized their knowledge to make comprehensive tools. He thanked NIOS for the hospitality provided.

NIOS faculty also provided their feedback. Mr. Sanjay Kumar Sinha, RD Patna felt that workshop design facilitated deeper introspection and provided a platform to share insight and bring about out improvisations. Mr. Ravi Shankar, Joint Director Media expressed his gratitude for the acknowledgment of media resources and thanked NIOS for giving the opportunity to spread the role of media. Dr. Sandhya Kumar D.D Academic also sought clarity on the issue of weightage to TMA.

14 | P a g e  

 

Dr. Kuldeep Agarwal appreciated all the participants for their contribution in reviewing the Monitoring protocols and thanked Prof. Madhu Parhar for her valuable input regarding developing a framework of monitoring and evaluation. He also thanked Mr. Dharuman for his support and appreciated the efforts put in by AEP Unit.

Speaking about the workshop Mr. Dharuman said that he had observed the sessions from time to time and was very impressed at the kind of participation the sessions were able to elicit. He felt happy about the learning’s from the workshop and proposed to finalize the handbook so that it would be presented to Academic Council for its approval.

The Vote of Thanks was delivered by Ms. Asheema Singh, Project Coordinator (AEP)

Conclusion

All the expected outcomes of the workshop were achieved. The participants seemed to enjoy both the sessions and the learning approach used by the facilitators. They were very active in all sessions, showing great interest and enthusiasm. Workshop Feedback Form was given to all the participants. All participants commented that the workshop had been very helpful and all the objectives were satisfactorily achieved and that they were glad that NIOS was taking such initiatives.

15 | P a g e  

 

Resources Material

1. Accreditation of Institutions as Study Centre of NIOS: Norms and Procedures 2. Standard Operating Procedures for Study Centres (AIs) of NIOS 3. Standard Operating Procedures for Personal Contact programme (PCP) and Tutor

Marked Assignment (TMA) 4. Guidelines for Personal Contact Programme (PCP) 5. Performa of Inspection Report for Grant of Accreditation 6. Handbook for Academic Facilitators of NIOS

List of Participants

1.Mr. C. Dharuman, Director, SSS, NIOS 2.Dr. Kuldeep Agarwal, Director, Academic, NIOS 3.Prof. Madhu Parhar, STRIDE, IGNOU, Delhi 4.Mr. Sanjay Kumar Sinha, Regional Director, Patna 5.Mr. Ravi Shankar, Joint Director, Media, NIOS 6.S.K Prasad, SAP, NIOS 7.Ms. Bhawna Dhyani, Deputy Director, NIOS 8.Dr. Mamta Srivastav, Deputy Director, NIOS 9.Mr. R. D. Mahapatra, Deputy Director (Evaluation) 10. Dr. Sandhya Kumar, Deputy Director, NIOS 11. Ms. Neena Golani, Assistant Director, NIOS 12. Mr. P.M. Pankaj, Assistant Director, NIOS 13. Ms. Rachna Bhatia, Assistant Director (Acad.), NIOS 14. Mrs. Kirti Sharma (PUNE) 15. Dr. Kiran Angra,Principal (Chandigarh) 16. Mr. G.V.Shastri, Principal retd. KVS/NVS(Hyderabad) 17. Sh. Subrata Acharya (Kolkata) 18. Mr. Narendra Singh, (AI NO.210,210) (Delhi) 19. Ms. Asheema Singh, Project Coordinator (AEP) 20. Dr. Rajendra Nayak, AO (mathematics),NIOS 21. Ms. Radhika, AO (ICT) 22. Mr. R.S. Rana, Principal retd. KVS, NIOS 23. Ms. Ankita Juneja, SEO(AEP), NIOS 24. Ms. Shveta Gupta, SEO(AEP), NIOS 25. Ms. Sheetal Gupta, Intern, CIE, Delhi 26. Mr. Shrey Malik, Intern, Sushant School of Design Gurgaon 27. Mr. Sunder Singh Rawat, Office Assistant, NIOS