Overview and Call for Proposals

Grant Applications are now available here.  However, please READ all of text below too!

2017 applications can be submitted at any time and will go out to reviewers, whose judgments will be combined into a final decision in about 4-5 weeks.

 Overview—“Why 3F?” 

This fund (3F) was established in order to facilitate and accelerate the develop­ment of the theory, practice, and profession of evaluation. It is based on the belief that explicit attention to critical study of the methodology and foundations of a discipline and its applic­ations—especially in the case of an emerging or radically changing discip­line—can avoid many dead ends in its development, and nurture valuable new perspectives. These improve­ments, in the case of evaluation, are not merely academic refinements, because the whole operation of society depends critically on the careful identification and demon­stration of the success or failure of its efforts at improvement and its response to crises, i.e., on ethical professional evaluation. Funding from 3F is allocated with signifi­cant weight given to social payoffs from evalua­tion for those in need, including uncon­ventional needs for which a case can be made.

Evaluation itself had to spend fifty years to achieve a moderate degree of legitimacy after its discard, at the beginning of the twentieth century, into the trash heap of scientifically untouch­able topics, a rejection based on a superficial and fallacious critique of evaluation by the posit­ivists. That indefensible blunder meant that many, perhaps most, social scientists turned their backs for half a century on full frontal attacks on the great problems faced by our global society, a decision which almost certainly cost us all dearly. 3F is an attempt to reduce the chances of similar mistakes and similar costs. Its modest resources are devoted to supporting proposals for new app­roach­es to the task of extending the domain of rational scientific efforts at objective analysis of eval­uative issues at either the applied or theoretical level, including methodological or foundational issues.

In this effort, 3F’s aim is to generate the development of new perspectives on and applications of evaluation, with some preference to ‘out of the box’ or ‘long shot’ projects and proposals that are likely to find funding hard to get from the usual sources because of their departure from the current research paradigms and/or their relatively low chances of success. Comments, from a number of Nobelers amongst others, have supported the need for this kind of approach as filling a gap in the current research funding portfolio across all research disciplines, reaching far beyond the sciences, e.g., into jurisprudence, technology, mathematics, history, literature, and the classical arts.

Goals. The 3F approach is intended to be, and should be seen as, simply complementary to the great efforts made by the American Evaluation Association (AEA) and similar professional organizations world-wide. In particular, while the AEA covers a vast range of support for professional evaluation members and their needs, including an excellent system of honorary awards for research and service, our intent is more narrowly focused, specifically research intended:

  • to generate new research rather than reward completed research;
  • to focus on a certain sub-area of research (which we might call normative meta-research), which is just a small but crucial part of the big field of research on evaluation itself;
  • to facilitate the emer­gence of new or massively transformed paradigms, critiques, and pract­ices, rather than the mere refinement of existing ones, even excellent ones; and
  • to emphasize continued consideration of the potential social benefits of refining evaluation theory, practice, and methodology.

Structure. 3F is set up as a donor advised fund under the umbrella of the Marin Community Foundation, the third largest community foundation in the US (in US legal terminology it is a nonprofit 501(c)(3) organ­ization). That management and funding is set up to continue in perpetuity, subject to the continued supervision of the 3F Advisory Committee, a group of experienced evaluation specialists and disciplinary leaders representing many evaluation approaches, including many ex-presidents and editors of eval­uation organizations and publications such as our leading journals and anthologies (their names and interests can be found at the 3F website FasterForwardFund.org; Jane Davidson is our Chief Operations Officer).

Ongoing 3F Projects 

3F Annual Grants—2017

First, remember that every entry must be eligible as a “3F Project”.  To get a better understanding of possible eligibility, we will post online a few projects funded last year. (After completion, we will post short summaries of 3F funded grant projects with links to more extensive summaries or the actual completed projects, as appropriate on our website.) This year, in conjunction with our regular General Topic grants, we have added 3F Focal Topic Grants as possible areas for exploration (described in more detail below*). For further clarification of what a 3F General Mission project might be, here are some ideas that illustrate the possible topical range of a 3F project:

  • examining the evaluative assumptions and methodology used by historians of warfare and warfare technology in supporting evaluative judgments;
  • the scope and limitations of ‘big data’ methodology for social policy analy­sis;
  • supporting work by a small international specialist online group focusing on specific technical problems in intercultural cooperation and evaluative comparis­ons;
  • the application of game theory (or positive psychology; or addiction theory) to the ethics of suicide prevention;
  • the quality of a sample of evaluations done by a state’s legislative analysts (or by a federal inspector-general’s office; or by the World Bank); i.e. meta-evaluation;
  • the utility in certain circumstances of what might be called ‘televal’ by analogy with the now highly active field of telemedicine, e.g., to bring technical or manage­ment skills from highly experienced evaluators to Nepal or Namibia or the Northern Sioux;
  • developing new curricula and pedagogy for teaching evaluation method­ology in the K-16 realm (or in the professional development area);
  • the validity of quality-adjusted life years (QALYS) as the emerging key metric for national/­inter­national social (or medical) interventions, or an analogical problem in educational research;
  • simplified public access/participatory (P.A.P.) evaluation systems such as Emily’s List, Amazon reviews, JD Powers, US News & World Report (see Foci list below).

In general, supported proposals will undertake to produce new understand­ing or evaluative information about, or methods for doing, evaluation, in a form suitable for immediate publication, typically in a paper, chapter, or a set of these; or perhaps using augmented reality video technology.

Meta-evaluations are of course eligible (see bullet 5 above), but support depends on their methodological significance, e.g., to improve the logic of rubric formulation.

What’s not typically funded. It is unlikely that funding will provide support for purchase of travel or durable equipment, since online video and technology rental are usually adequate; and of course, work that has been previously published, online or in hard copy, will not be eligible. Also ineligible is any evaluation of the usual applied kind that currently occurs in the score of fields where thousands of professional evaluators already work, not even if it uses some novel methodology; the research must be on that novel methodology, not just embodying it.

Funding amounts. The financial limitations on 3F support at the moment mean that applications for a few thous­and dollars are perhaps slightly more likely to be supported than those requesting or requir­ing tens of thousands (partly because there can be more of the former); and requiring hundreds of thousands puts a proposal beyond the current 3F range. This year’s funds available will be in total, approximately double those provided last year, but individual grants will most still be about the same (i.e. mostly small; under 10K). We can be useful as co-sponsors for larger (and smaller) projects since that approach means many proposals can be funded for a lesser amount than requested due to our financial limitations. Running big evaluations is often expensive; inventing valuable new methods or critiques is often very important and not so expensive.

2017 Proposals to 3F  (click here to download an application).

The current requirements are fairly simple:

  • describe what you and your group/organization is and wants to do, when, where, and how. Include methods to be used, materials and services required and a timeline. (Timeline must note a proposed mid-point task(s) and/or indicators and a due date for them; as well as a completion target date, usually less than 12 months.);
  • describe indicators you think should be used to assess progress and conclusions of the project and note these in your timeline;
  • provide an argument, and perhaps evidence (e.g., a needs assessment), that this effort is consistent with the aims of 3F. This should include a discussion of social payoff and epistemological benefits (i.e. knowledge + understanding/insights & behavior);
  • give your estimate (and evidence) of feasibility and importance, with emphasis on expected tangible results that will be widely available; include qualifications of the personnel involved (note expertise, experience, knowledge, training, etc.).
  • submit a detailed budget and, if appropriate, evidence for its accuracy and cost-effective­ness (e.g., don’t include travel when video can do the job). Note that 3F will not pay overhead beyond about 10%, preferably less.
  • keep all this down to a maxi­mum of a 3-5,000 thousand words (e.g. by severely condensing your cv), to reduce the investment of time by you and us. (It is acceptable to point to a website which has additional information about you or your project. However, the website information will not necessarily be read, so the submitted proposal must contain sufficient information to be complete and stand on its own.)

Budget suggestions. There are two considerations to keep in mind that distinguish 3F from some–perhaps most–other funding agencies.  First, the reviewers, other things being equal (i.e., assuming equal probability of success and equal importance of a successful outcome), will be likely to fund less expensive projects over more expensive ones, since they are trying to maximize the payoff for the discipline and society from a limited budget of their own. So the common practice, in proposal budgeting, of multiplying your normal salary rate by the number of days or weeks you think the work will take, may make you uncompetitive with someone, whose project is otherwise equally promising, who sets their payment at the minimum level that will get them to take on the task. Second, reviewers will not proportionately penalize long-shot projects by comparison with those whose probability of success is higher: they will give substantially greater weight to the payoff compared to the expectation of success.

Proposal deadline. 2017 applications can be submitted at any time and will go out to reviewers, whose judgments will be combined into a final decision in about 4-5 weeks.

While we strongly recommend using colleagues for extra input and critique of your proposal, they should not be identified as co-authors for that kind of input. A principal investigator (or at most 2) and financial point person, must be identified to take responsibility for the project and its funds.

Financials. Payment to successful applicants is best done via a legally qualified philanthropy, e.g., your univer­sity or organization (keeping in mind that 3F will normally expect to pay no more that 10% overhead, and see here for possible overseas funding information needed). Payment will normally be in three installments: a small sum for start-up costs; a mid-project payment; and a final payment (after completion of project and acceptance of final report, including a 150-200 word summary). Reporting 3F results openly, which we think is important, means that a summary of your project and perhaps a link to your final project may be posted on the web and perhaps elsewhere.

There is no hidden agenda in the sense of a preferred approach other than that outlined here (for a copy of our current rubric visit FasterForwardFund.org/. . . ). The actual judging for awards, and all other strategic management decisions, is overseen by the 3F Advisory Committee.  Decision time will depend on the number of entries, but we’ll try to keep it between 4-6 weeks.

Feedback  Evaluators should practice what they preach, and there are good reasons for that practice, so we welcome suggestions for improving this proposal or its presuppositions. Also, we are always working to improve transparency, and have a copy of our draft rubric for 2016 here. Send comments to our email: FasterForwardFund@gmail.com with the subject: Suggestions.

 *New 3F Project-The Faster Forward Fund Special Focus Prizes (4F):

The Essay Prizes. A new $1,000 “Special Focus Award” will be given for each of the five categories below (assuming minimal standards are met). Each of the five following topics will be run as a separate competition.

Written entries must be submitted with less than 1,500 words, with no names or other identifying information in the body of your submission.  Your name and contact information must be included on a separate cover sheet and in the body of your email, which should have the subject line: 3F Foci # (insert focus number); no cv or nonprofit affiliation necessary.

Send entries to: FasterForwardFund@gmail.com by midnight PST on 15 November, 2016 (an extended deadline).

Special Focus 1  – Ethics & Scientific Method

Do reason and scientific method oblige us to adopt and report our evaluations from an ethical point of view; or do they oblige us to be skeptical about or to ignore ethical considerations? Note: this means evaluating the ethical defensibility of the evaluand itself, not the, or not just the, methodology and creation of the evaluation. In other words is ethics simply one branch of applied science? (A 2,500 year old problem; time to solve it?)

Special Focus 2 – Peter Singer Prize

Best design for the evaluation of at least three charity watchdog organizations; e.g., Charity Navigator. These outfits are the primary evaluators used by donors of millions of dollars every year. Which of them gives the best advice is therefore a really important question and 3F should try to provide a good meta-evaluation. Note: problems of data access and transparency of results should be addressed, and a cost estimate included. An effort should be made to provide for a low cost (less than 25K) and a higher cost (50K) approach. You can also indicate your interest in doing the evaluation (with further funding) as yes or no.

Special Focus 3 – Meta-evaluation Checklists

Best critique of available meta-evaluation checklists, including at least those by Dan Stufflebeam, GAO, and Michael Scriven. Extra points will be awarded for the best improved version based on or unlike the published ones (in other words, a prize for the best invention and use of a meta-meta-evaluation checklist).

Special Focus 4 – Meta-evaluation of Mainstream Science

At long last, serious work has begun on investigating whether the hard-core sciences are in fact scientific, at centers in Stanford and the University of Virginia. Their first reports are out and both are very critical. The first meta-meta-evaluation is also out and is very critical of UVA’s effort. Can we improve this area of research?

Special Focus 5 – Future Topic Ideas

Suggest and support a special focus 3F topic or topics for 2017. What would be useful to accelerate the discipline or practice of evaluation (i.e. take us faster forward)? Why is it important? What problems is it causing because it is not being adequately addressed?

*Project Support Awards. If you would like to actually attempt to address one of these issues, please submit it as a proposal for regular 3F General Mission funding rather than as an idea or approach for someone else to address next year. (See earlier requirements listed and visit FasterForwardFund.org for complete details.)

A slight preference will be given to 2017 general grant applications that tackle one of the foci on this list—this means if two submissions of equal merit are entered, and only one can be funded, preference will be given to a 2016 special foci topic submission.

 Other 3F Projects:

Project FFF/1, the Micro Pro Bono Project. Many Advisory Committee members have volunteered to offer some unpaid email, phone or video micro-services to those who need them. Specifically, if you send in a request to FasterForwardFund@gmail.com, headed “Micro Pro Bono Request” explaining what kind of advice about evaluation you need, why you need it, and who on the Advisory Board you’d like to get it from, then we’ll forward it to that person—or to an alternative person that you or we suggest, if that’s necessary to control overload.

Your adviser or our staff will then email you a time window when you can call your adviser on a free service like Skype. Our advisers can typically give advice about specific quest­ions, e.g., how to frame the high-level evaluation questions you’ll need to address, what to look for in an evaluation team with the skill sets to answer them; understanding the main alternatives you should consider as evaluation approaches; ballparking the kind of time or budget allowance for evaluation services that might be approp­riate; what ‘program logic’ means; what to do when you don’t have baseline data for the time when the intervention began. The donors are not offering to be a continuing source of help, just a ‘get-started’ or ‘crisis-helping’ source, so they can’t be assumed to be willing to talk to you more than once, or for more than an hour on that occasion.

Our main idea is simply to help people think­ing about using evaluation to be realistic about what that involves or might yield. (But doing this may set an example that could expand e.g., into a Pro Bono Registry, or at least be replic­ated by other groups.) And don’t assume that specific suggestions from one individual, either in the content of this document or in advice from any of our pro bono advisers are guarantees of support from 3F as a whole, which of course would require specific discussion by the whole Advisory Committee.

The FFfellowship project at Claremont Graduate University. In a separately funded project, a $5,000 prize is offered each spring for the best essay on a 3F theme of the applicant’s choice by a graduate student in evaluation at Claremont Graduate University. We receive many diverse entries—the prize winning entries are posted at FasterForwardFund.org where entry requirements and deadlines are also posted. (Also a small prize is funded by 3F for the best PhD thesis of the year done at any university. It is administered by the Evaluation Program at Western Michigan University.)

The 3F Book ProjectEvaluation for an Equitable Society was published in 2016 and was inspired by the 2014 European Evaluation Society Conference. You may purchase this book at Amazon. Other book projects are in the works including the Supertext project (see below).

The Critical Thinking Supertext Project. This text, both theoretical and practical in design, will focus on the furtherance of critical thinking as a discipline, rather than an evaluation as such. The detailed production and administration of this, which—like its funding—is separate from 3F, is in coordination with the University of Windsor and the international critical thinking association. Updates will be posted on our website. (Critical thinking is an essential entry-level skill for the practice of evaluation, perhaps the most important one.)