FUNDAMENTAL RESEARCH GRANT
APPLICATION & EVALUATION
Muhammad Fauzi Mohd ZainFakulti Kejuruteraan dan Alam Bina, UKM18 Feb 2014
1 Introduction
1 Introduction
Program Penyelidikan Fundamental
2013
Single Disciplinary Project
-permohonan projek penyelidikan melibatkan satu (1) bidang penyelidikan.-siling peruntukan adalah RM250,000.00 sepanjang tempoh penyelidikan.
Trans Disciplinary Project
- permohonan projek penyelidikan melibatkan sekurang-kurangnya tiga (3) bidang penyelidikan (trans disciplinary) melangkaui jabatan / fakulti dari institusi yang sama.- sasaran hasil penyelidikan yang sama.- siling peruntukan adalah RM1,500,000.00 sepanjang tempoh penyelidikan.
Research Acculturation Grant Scheme (RAGS)
-dana tunas yang bertujuan untuk membudayakan penyelidikan di kalangan penyelidik muda di IPTA bukan RU sebagai persediaan untuk membangunkan prestasi penyelidikan supaya dapat berdaya saing di peringkat kebangsaan dan antarabangsa.
Research Acculturation Collaborative Effort (RACE)
Membantu Non-RU meningkatkan budaya penyelidikan dan seterusnya dapat meningkatkan output hasil penyelidikan. Usaha ini dapat mempercepatkan Non-RU untuk mencapai tahap setanding dengan RU yang lain.
1 Introduction
BIL PERKARA
FRGS
LRGS ERGS PRGSSingle Disciplinary
Project
Trans Disciplinary
ProjectRAGS
1 Siling Permohonan RM250,000 RM1,500,000 RM50,000 –
RM80,000RM3
juta/tahun RM300,000 RM500,000
2 Tempoh Penyelidikan
1 hingga 3 tahun
1 hingga 3 tahun
1 hingga 2 tahun
3 hingga 5 tahun 3 tahun 2 tahun
3 KPI • 1 PhD• 3 papers in index link journal (2 years)
• 10 PhD (3 years) • 50 papers (3 years)• 3 IP (per program) - number of researchers with Citation Index of 100)
• 1 PhD• 3 papers in index journal• 1 IP (filed)
•1 IP/project
1 Introduction
Penyelidik
RMC IPT
Panel Penilai KPM
Jawatankuasa Induk FRGS
Peringkat IPT
Peringkat Nasional
2 ahli panel membuat penaziran
RMC membuat proses saringan terperinci dengan bantuan pakar dalaman/luaran
60 pakar
Meluluskan polisi, halatuju dan dana
Panel Penilai IPT
Jawatankuasa Penilaian FRGS KPM
1 IntroductionCARTA ALIR PROSES KERJA RMC DI PERINGKAT CARTA ALIR PROSES KERJA RMC DI PERINGKAT
UNIVERSITI UNIVERSITI
1 IntroductionCARTA ALIR PROSES KERJA RMC DI PERINGKAT CARTA ALIR PROSES KERJA RMC DI PERINGKAT
UNIVERSITI UNIVERSITI
1 IntroductionSENARAI DOKUMEN YANG PERLU DISEDIAKAN SENARAI DOKUMEN YANG PERLU DISEDIAKAN
1 Introduction
Fundamental Research (FRGS) Basic research Pure research
Fundamental research generates new knowledge and technologies to deal with unresolved problems.
1 Introduction
Exploratory Research (ERGS) Relies on secondary research To gather preliminary information
Research intended only to provide greater familiarity with the phenomena that researcher wants to investigate so that he can formulate more precise research questions and perhaps develop hypotheses. Such studies can be essential when researcher is investigating new phenomena or phenomena that have not been studied before.
1 Introduction Most common reasons for grant writers
(GWs) not receiving funds1. Not new or lack of original ideas2. Diffuse, superficial or unfocused research
plan3. Lack of knowledge of published relevant work4. Lack of experience in the essential
methodology5. Uncertainty concerning the future directions6. Questionable reasoning in experimental
approach7. Absence of acceptable scientific rationale8. Unrealistically large amount of work9. Insufficient experimental detail10.Uncritical approach
1 Introduction
Quality of the Proposal The measures for a good quality proposal
are:
Informative title; Convincing executive summary; Clear problem statement; Scientific background and rationale; Good selection of research methods; Ethical considerations; and Realistic budget and schedule.
1 Introduction
A document that is neat, well organized and easy to read;
Responsiveness to the program need, with specific references showing how the proposed project will achieve program goals and objectives;
Fresh insight into an important problem; Writing that communicates the enthusiasm and
commitment of the researcher; Evidence that the PI knows the field; Convincing preliminary data; and A feasible work plan that is supported by an
appropriate budget.
Characteristics of a good proposal:
2 Evaluation: Evaluator
Internal evaluation Department/Faculty/Institute level University level
Internal evaluators improve the quality of grant submissions
How to “improve”
2 Evaluation: Evaluator
External evaluation Sponsor level Many constraints
Number of proposals, amount of money available, etc.
To find mistakes/weaknesses, etc.
To find ways on…
How to “reject”
2 Evaluation: Evaluator Selecting An Evaluator
When selecting an evaluator for research grant project, the following criteria may be useful to consider:
Credentials/Reputation- To what extent has the individual evaluated research grants, particularly fundamental focused researches? Is the evaluator affiliated with an academic institution?
Education- To what extent the individual trained in evaluation? Does the evaluator have a certificate, or doctoral degree related to evaluation?
Experience- To what extent does the individual have formal or informal experience with evaluation in fundamental research grant? Can the evaluator provide samples of evaluation reports that he or she has completed?
Sensitivity- To what extent does the evaluator have experience working with the target population?
Integrity- To what extent is the organization familiar with the evaluator? Does the evaluator present any conflicts of interest with evaluating the project?
2 Evaluation: Evaluator Selecting An Evaluator
When selecting an evaluator for research grant project, the following criteria may be useful to consider:
Communication Skills- To what extent is the evaluator able to explain technical concepts in understandable language and demonstrate clear verbal and written expression?
Availability- To what extent is the evaluator available to meet with the project timelines and be flexible if timelines need to be modified?
Cost- Are the proposed cost for the evaluation reasonable for the required tasks?
Contract/Scope of Service- Is the evaluator willing to design a scope of service/contract or agreement outlining his/her responsibilities along with timelines?
2 Evaluation: Evaluator Pros and Cons of using External Evaluators
Pros Cons
Less work for your organization
Evaluators professional expertise
Evaluators bring objectivity
Evaluation results may have more credibility
Less control over the process
Staff may have more complete understanding on the program
Less opportunity to develop internal evaluation capacity
Expensive
2 Evaluation: Evaluator Conflict of Interest
Evaluators are required to declare any personal interests according to the following criteria. Evaluators must disqualify themselves if they can in any way benefit from the approval or rejection of the proposal. They must also disqualify themselves in the following circumstances: Evaluators have close collaboration with the GW (e.g. have co-authored and published an article with the GW during the past three years, have been involved in the preparation of the application, or are involved in the publication or application of the results) Evaluators have been a superior, subordinate or instructor of the GW during the past three years Evaluators are currently applying for the same post as the GW Evaluators are currently applying for funding from the same funding instrument on the same research area.The GW is a close person to evaluator.
3 Evaluation: Process Research Proposal VS Research
evaluationModule of FRGS/ERGS Proposal
TitleDetails of ResearcherResearch InformationExecutive SummaryResearch Background
Problem StatementHypothesesLiterature Review
Research ObjectivesMethodology/Research DesignTimeline/ScheduleExpected ResultsFacilities and Special ResourcesBudgetResume/Brief CVAppendices
What Ext. Evaluators Look For:
Title (1)Details of ResearcherResearch InformationExecutive Summary (2)Research Background
Problem Statement (3)HypothesesLiterature Review
Research Objectives (4)Methodology/Research Design (5)Timeline/ScheduleExpected Results (6)Facilities and Special ResourcesBudget (7)Resume/Brief CVAppendices
Fundamental Criteria of Evaluation
CARE: Are GWs tackling an important problem? If they can make progress on it, will anyone care?
NOW: Why now? If this problem is so important, why has it not been addressed before?
IDEAS: Do GWs have concrete ideas for starting an attack on the problem and a vision for proceeding further? Is initial progress likely and subsequent progress possible?
RESULTS: Do GWs have some preliminary results? Do they demonstrate a good understanding of the problem and the methods needed attack it further?
PLAN: Do GWs have sensible plans and methods (e.g., concrete steps and ways of decoupling risks)?
CAN-DO: Why these GWs? Why are their qualifications and infrastructure appropriate?
LEGAL: Have GWs followed the rules of the solicitation (e.g., compelling broader impacts for FRGS/ERGS)?
.
3 Evaluation: Process
Main Criteria of Evaluation All proposals must be evaluated based on the criteria outlined below.
Relevance Degree to which the proposal was related to KPM (F/E/L/PRGS) Degree to which the proposed research results would be
relevant to policy-makers Extent and appropriateness of dissemination plans Degree to which the proposed research would facilitate the goals
of KPM Degree to which the proposed research represented an innovative
approach and develop new knowledge in the field of engineering or other disciplines
Research Team Quality of the research team and their research The applicant and the research team are among the leading in their
field. The publications are at a remarkable international level. The articles are published in the best peer-reviewed journals, or proceedings, which are indexed in the leading databases of the field.
The impact of the applicant (number of citation; the scientific level of the journals, where the articles are published) is, in the respective field, at a remarkable international level)
3 Evaluation: Process
Main Criteria of Evaluation
All proposals must be evaluated based on the criteria outlined below. Proposal Quality
Degree to which the research approach provided results that can be generalized beyond the unit of analysis and were applicable to multiple settings or groups
Degree to which proposal demonstrated general criteria of excellence
Innovation Quality of the research objectives and linkage to
literature review Clarity of the research questions Appropriateness of methodology Quality/clarity and detail of proposed research method etc
3 Evaluation: Process
Main Criteria of Evaluation (detail)Evaluating what worked and what didn't will be crucial for grant sponsor and for GWs. What impact do evaluators expect to achieve and how will they evaluate it? Before evaluators design their evaluation, Carlson and O'Neal-McElrath, authors of Winning Grants, Step by Step, suggest that evaluations can accomplish these 6 purposes: To find out if the hypothesis was right. Did GWs actually do what they set out to do? To determine if the methods specified were used, and if the objectives were met. To find out if an impact was made on the identified need. To obtain feedback from the people served and other members of the community. To maintain control over the project (evaluations are done at various points in the project). To make changes in the program mid-stream, if necessary, to insure the program's success.
3 Evaluation: Process
Determine if GWs will use quantitative or qualitative methods for their data collection, or what combination of the two types they will use. Develop a good description of these methods and their rationale for the grantor.
Make sure the evaluation component of GWs proposal connects with the proposal's objectives and methods. If those objectives and methods are measurable and time-specific, the evaluation will be easier to design.
Evaluators must ask these questions as they develop the evaluation section of GWs proposal: What is the evaluation's purpose? How will GWs use the findings? What will GWs know after the evaluation that they
didn't know before? What will GWs do as a result of the evaluation that
they couldn't do before because they lacked the relevant information?
How will the clients and community served be better as a result of the program?
3 Evaluation: Process
Main Criteria of Evaluation (detail)The following criteria will be used to evaluate proposals:Quality of the proposal (i.e., significance, relevance of methodology, importance of findings, value to larger academic community, etc). Proposals should be framed in a manner easily understood by someone outside your discipline. Proposals must meet high professional standards in presentation, including spelling, grammar, and proper budgeting.Potential for peer-review publication, presentation, exhibition, etc.Budgetary appropriateness.Potential for disseminating research to broader academic community.Value for professional development of faculty member.
3 Evaluation: Process
Main Criteria of Evaluation (detail)Review questions include, but are not limited to:
Does the proposal communicate the importance of the work?Is the importance of the project within its field made clear?Will successful completion of the project have an impact upon the field?Is the project significant to the development of a program of scholarly activity by the lead GW?Are the objectives clearly defined, and is the basic question to be answered clearly identified?Are all necessary facilities available?
3 Evaluation: Process
Main Criteria of Evaluation (detail)Review questions include, but are not limited to:
Does the lead GW have the necessary background and expertise?Are project activities well planned, and do they realistically fall within an appropriate timetable?Have all items requested been justified?Is the amount requested reasonable and consistent with the total funding available to this grant program?If equipment is requested, has the possibility that it is already available elsewhere on campus been addressed?If support for students is requested, is it clear that their activities are essential to the research program?
3 Evaluation: Process
External evaluation Sponsor level Evaluators focus on the Four Cs
Clarity. How GWs do cross-reference current literature in laying out their premises.
Content. How GWs organize their ideas around aims linked to their hypothesis.
Coherence of concepts. How GWs present coherent set of ideas predicated by previous work.
Cutting edge. Are GWs ready to take legitimate risks.
3 Evaluation: Process
4 Evaluation: New Approach
5 Evaluation: Title
A good title should:
Indicate the type of study. Address the main problem. Be concise, short, and descriptive. Convey to the evaluator the main focus of the research. Use the correct terms in the title.Should be intelligible to non-specialists.Limit the title to a single sentence.Relevant in 2 years time?
5 Evaluation: Title
Selection of research topic should be based on…..
Magnitude of the problem and its impactUrgency of the need for a solution.Relevance to the aim of the funding agency.Amenability of the problem to investigation.Feasibility of the approach.The proposed research topic is very timely and relevant both internationally and locally. The prospective results make a substantial contribution to the development of science,
technology, and/or society
6 Evaluation: Executive Summary
An informative abstract, giving evaluators the chance to grasp the essentials of the proposal without having to read the details
GWs must present their project ConciselyState significance ClearlyState Hypotheses, Research Problem, SolutionMethods and RationaleExpected output.
7 Evaluation: Research Background
Research Background (RB)
1.Title2.Problem statements3.Objectives
Flows naturally from Title, Problem Statement to Research Objectives
7 Evaluation: Research Background
The RB serves several important functions: Evaluators must ensure that…
GWs are not "reinventing the wheel". GWs demonstrate their knowledge of the research problem. GWs demonstrate their understanding of the theoretical and research issues related to their research question. GWs show their ability to critically evaluate relevant literature information. GWs indicate their ability to integrate and synthesize the existing literature. GWs provide new theoretical insights or develops a new model as the conceptual framework for their research. The proposed research will make a significant and substantial contribution to the literature (i.e., resolving an important theoretical issue or filling a major gap in the literature).
7 Evaluation: Research Background
Problem Statements The most important aspect of a research proposal is
the clarity of the research problem The problem statement is the focal point of the
research Evaluators must ensure that…
GWs give a short summary of the research problem that have been identified.
The research proposal may not acceptable or credible if GWs not clearly identify the problem.
GWs present the persuasive arguments as to why the problem is important enough to study or include the opinions of others (politicians, futurists, other professionals)
7 Evaluation: Research Background
References-Most resent
Up-to-dateHighly relevant with the problemOriginal source
First Order : High Impact Journals and Books
Second Order : Indexed Proceeding Publications
Third Order : Reputable Technical Report
8 Evaluation: Objectives
Objectives specify the outcome of the project, the end product(s). GWs must state the objectives clearly and keep them “S-M-A-R-T” or “S-I-M-P-L-E.“ Specific -what GWs intend to change through their project.Immediate -time frame during which a current problem will be addressed.Measurable -what GWs would accept as proof of project
success.Practical -how each objective is a real solution to a real
problem.Logical -how each objective systematically contributes to achieving GWs overall goal(s).Evaluable -how much change has to occur for the project to be effective.
9 Evaluation: Methodology GWs state it explicitly GWs give an overall summary of the research design
and methodological approach. GWs provide the methodology for each specific
objective. GWs describe
the specific design (what will they do and how, number of replicates, etc.),
the materials and techniques that will be used, and
the feasibility of these techniques.use literature to support design, materials &
techniques
Walk the reader through GW projectDescribe the activities as they relate to the
objectivesDevelop a time line and/or and organizational
chartHow will the activities be conducted?When?How long?Who?Where?What facilities?
Basically, GWs must provide answers to the following questions: What activities need to take place in order to meet the objectives? What are the start and finish dates for the activities? Who has responsibility for completing each activity? How will participants be selected? What factors determine the suitability of your methodology? Does this project build on models already in existence? if not, how is it superior? What facilities and equipment will be required to conduct the activities?
9 Evaluation: Methodology Milestones
The milestones are the results which The milestones are the results which the project seeks to achieve. the project seeks to achieve. The milestones should, as much as The milestones should, as much as possible possible relate to ‘tangible productsrelate to ‘tangible products’ ’ (quantifiable, qualitative or verifiable) (quantifiable, qualitative or verifiable) from conduct of the research. from conduct of the research. They They indicate viable achievementsindicate viable achievements..
9 Evaluation: Methodology
Gantt’s Chart /Flow Chart
GWs must clearly show the research activities and milestones (•/M)Reflection of the project objectives,
methodologies, outputs, etc. Very important
10 Evaluation: Expect Output
Scientific OutcomesNew methodologyPatentsPublications (High Impact Journal)Discoveries
Socio-Economic outcomes/impactBetterment of SocietyDevelopment of the EconomyImproving Livelihood of People
What are the expected outcomes and what do GWs wish to achieve, e.g.:A new theoryA prototypeA new modelAn artefactA new plant processA solution to a practical problemA specific aid to practitioners in a particular fieldAn instrument of use in the manufacturing industry, etc.What contribution will this research make to the body of knowledge in the particular field of study?The F/EGRS Evaluation Guidelines specifically require that the expected outcomes be clearly defined, as well as the likelihood that the research will achieve the expected results within the stated timeframe.
11 Evaluation: Track Record
Track Record (CV) Experience, Qualifications and Availability of Research Team
This section should begin with the principal investigator, and then provide similar information on all individuals involved with the project. Two elements are critical:Professional research competence (relevant research experience, the highest academic degree held, and technical societies).Relevant management experience (if any).
12 Evaluation: Quality of Proposal
Style:Use most recent formFollow guidelines (font, size, margins, etc.)Spell check, correct grammarHighlight signposts (italic, bold, underlining)One main idea per paragraphUse topic sentencesUse transitions (e.g., in contrast, however, likewise, etc)Use graphics in methodology and needs sectionsEnd paragraph with closing sentence
Evaluator-friendly applicationGWs must give evaluators enough time !
12 Evaluation: Quality of Proposal
Novelty, Cutting Edge, High Impact
Does the research use novel techniques, tools, and procedures?Is new data required?Is data gathered in a new way?Is existing data utilised in a new way?Can an existing application be used in a new way?Is the proposed research potentially patentable and publishable?
13 Evaluation: Policy and Support
Budget (Grants Fail…)
Problem: The budget exceeds the available amount The budgeted items are not reasonable for the
work proposed The cost of the program is greater than benefit The budget justification is not included There is a mismatch between activities and
budget The budget justification does not clearly explain
the budget item Inappropriate use of funds
Solution: Outline budget items carefully and use standard amounts for expenses
13 Evaluation: Policy and Support
Budget: GWs must ensure that…
They present the budget based on the sponsor requests. (Read Guideline)The budget must be reasonable, acceptable, and appropriate (GWs must not inflate…)They must follow strictly the GuidelinesDetail justifications on each item must be provided (Vote 35000)
Itemized BudgetBudget Narrative
13 Evaluation: Policy and Support
Infrastructure/Facilities
Use whatever available in campus (related to proposed project)Reduce to a minimum any call upon outside facilities and expertiseThe requirements of infra will vary from study to study. GWs must carefully list the relevant facilities and resources that will be used. The costs for such facility use should be
detailed in GWs budget.
14 ConclusionMake Life Easy for Evaluators
Evaluators are knowledgeable, experienced scientists, but they can’t know everything.
Problem: evaluators may not get the significance of the proposed research.
Solution: GWs write a compelling argument. Problem: evaluators may not be familiar with all the
research methods. Solution: GWs write to the non-expert in the field. Problem: evaluators may not be familiar with the research
lab. Solution: GWs show to evaluators that they can do the
job. Problem: evaluators may get worn out by having to read
10 to 20 applications in detail. Solution: GWs write clearly and concisely, and make sure
the application is neat, well organized, and visually appealing.
14 Conclusion
Ethical Statement
Researchers undertaking any form of fundamental research using animals or people have to submit a proposal to either the animal ethics committee or the human ethics committee for approval before the data gathering can begin.
14 Conclusion Joel Orosz, of the W.K. Kellogg Foundation,
has sagely observed that there are really only four kinds of proposals
Bad idea, bad proposalBad idea, good proposalGood idea, bad proposalGood idea, good proposal
14 Conclusion
Good Proposal Grant Proposal (Failed/Success)-Sample
Good Ideas Good GrantsmanshipGood Presentation Good ReviewGood Luck
15 Bibliography
Kevin C. Chung, MD, Melissa J. ShauverCheryl Anne Boyce, Ph.DXander HT Wehrens, M.D. Ph.DGitlin, L. N., Lyons, K. J.Simon Peyton JonesBaharuddin Salleh
16 Ketua Panel Kluster Geran KPT1. Sains Tulin - Prof. Emeritus Dato' Dr. Muhamad bin Yahaya, UKM2. Sains Gunaan - Prof Emeritus Dato' Dr Md Ikram Bin Mohd Said,
UKM3. Sains Sosial - Prof. Dr. Samsudin bin A. Rahim, UKM4. Sains Tabii dan Warisan Negara - Prof. Dato’ Dr. Nik Muhamad bin
Nik Ab. Majid, UPM5. Sains Kesihatan dan Klinikal - Prof Dato‘ Dr. Amin bin Jalaluddin, UM6. Sastera dan Sastera Iktisas - Dato‘ Prof. Salleh bin Yaapar, USM7. ICT - Prof Dr. Ku Ruhana binti Ku Mahamud, UUM8. Teknologi dan Kejuruteraan - Prof Dr Muhammad Fauzi Mohd Zain,
UKM
Thank you