STATE OF NEW HAMPSHIRE Department of Education NH Statewide Assessments: ELA, Mathematics, Science RFP 2017-073 DOE New Hampshire Statewide Assessments RFP Questions Received by 12:00 PM on April 6, 2017 Additional Questions from the Vendor Conference Reference Question Response Page 4 of Cover Sheet, Contract Type: Not to Exceed $3M Please clarify if the “not to exceed $3M” encompasses the entire initial contract term of 3 years. $3M Annually Page 35, Section C-2 Requirements Table C-2 is included as an attachment to this RFP. C2 / p. 35 It appears Table C-2 was not included as an attachment to the RFP. Please provide direction on where to find Table C-2. Table C referred to on page 35 is the Excel Workbook attached to the RFP on the posting site. We apologize for the naming of the posted filed. It should read Table C. Cover sheet Please confirm that operational assessments are required for all three content areas (ELA, math, and science) for the 2017-2018 school year. Operational assessment for all three content areas (ELA, math, and science) are required for the 2017-2018 school year. Second page of RFP and… p. 33 From second page of RFP: “Vendors may bid on all five (5) components (components A, B, C, D, E), on any combination of assessment components (A, B, C, D), or any individual assessment component (A, B, C, D). For each component selected by the vendor, the proposal must address component E, and must address each component selected completely. The NH DOE will not accept bids for pieces of individual components.” and… From page 33: C-1 SCOPE OF WORK “The Scope of Work should be applied to each Assessment Component bid on:” Q: Are the above statements (red type) indicating that if a bidder wishes to bid on all five components, for example, they The bidder does not need to include five different sets of narratives if bidding on multiple components. Ex: If a bidder is bidding on the summative components for ELA, math and science, a single narrative could encompass the three subject areas. Ex: If a bidder is bidding on the summative and interim components for ELA, math and science, then the single narrative should ensure that any differences in the specifications of the summative and interim components be identified. This option was intended to give bidders an opportunity to present a proposal that may not be inclusive of all components. Ex: Summative option for ELA, math, science.
RFP 2017-073 DOE New Hampshire Statewide Assessments
RFP Questions Received by 12:00 PM on April 6, 2017
Additional Questions from the Vendor Conference
Reference Question Response Page 4 of Cover Sheet, Contract Type: Not to Exceed $3M
Please clarify if the “not to exceed $3M” encompasses the entire initial contract term of 3 years.
$3M Annually
Page 35, Section C-2 Requirements Table C-2 is included as an attachment to this RFP. C2 / p. 35
It appears Table C-2 was not included as an attachment to the RFP. Please provide direction on where to find Table C-2.
Table C referred to on page 35 is the Excel Workbook attached to the RFP on the posting site. We apologize for the naming of the posted filed. It should read Table C.
Cover sheet Please confirm that operational assessments are required for all three content areas (ELA, math, and science) for the 2017-2018 school year.
Operational assessment for all three content areas (ELA, math, and science) are required for the 2017-2018 school year.
Second page of RFP
and…
p. 33
From second page of RFP:
“Vendors may bid on all five (5) components (components A, B, C, D, E), on any combination of assessment components (A, B, C, D), or any individual assessment component (A, B, C, D). For each component selected by the vendor, the proposal must address component E, and must address each component selected completely. The NH DOE will not accept bids for pieces of individual components.”
and…
From page 33: C-1 SCOPE OF WORK
“The Scope of Work should be applied to each Assessment Component bid on:”
Q: Are the above statements (red type) indicating that if a bidder wishes to bid on all five components, for example, they
The bidder does not need to include five different sets of narratives if bidding on multiple components. Ex: If a bidder is bidding on the summative components for ELA, math and science, a single narrative could encompass the three subject areas. Ex: If a bidder is bidding on the summative and interim components for ELA, math and science, then the single narrative should ensure that any differences in the specifications of the summative and interim components be identified. This option was intended to give bidders an opportunity to present a proposal that may not be inclusive of all components. Ex: Summative option for ELA, math, science.
RFP 2017-073 DOE New Hampshire Statewide Assessments
RFP Questions Received by 12:00 PM on April 6, 2017
Additional Questions from the Vendor Conference
will need five different sets of narratives—each one covering most of the 45 narrative topics—and each of the five will need to apply the full “Scope of Work” to their response?
Or is there a different scenario intended?
Ex: Interim option for ELA, math, science. Ex: Summative and interim option for ELA and math only. Etc.
Pricing - 4th page before page numbers begin
Regarding the “not to exceed $3M” note on “contract type:” Please confirm that this is per-year.
Confirmed. NTE of $3M is per year.
4.17 / p. 11 - Organization Other than for the Executive Summary, is there to be any page-count limit—or guidance—for portions of the proposal, such as the 45 Narrative Topics in Section IV?
We did not want to put page limits. However we encourage clear and concise language leading to responses of reasonable length.
5.4.4 / p. 21 a. Please explain why this section talks only of the “Vendor-proposed software solution cost” getting allocated up to 24 points per bidder. Might it rather be the price of the total solution that somehow gets scored on the 24-point allocation?
b. Given that the terms of pricing for this RFP are a “not-to-exceed $3M”…on what basis will points be awarded to each bidder?
Vendor proposed software solution cost is inclusive of the total solution. The State will consider both implementation costs and subsequent year license and maintenance costs, provided in Tables F-1: Activities/Deliverables/Milestones Pricing Worksheet, F-4: Software Licensing, Maintenance, and Support Pricing Worksheet and, if appropriate, F-5: Web Site Hosting, Maintenance, and Support Pricing Worksheet. The cost information required in a proposal is intended to provide a sound basis for comparing costs. Factors include but are not limited to: Price Proposal • Cost effective budget. • Sound fiscal management
RFP 2017-073 DOE New Hampshire Statewide Assessments
RFP Questions Received by 12:00 PM on April 6, 2017
Additional Questions from the Vendor Conference
practices that meet or exceed industry standards. Criteria for these scores will be found in but are not limited to: Proposal Section III: Responses to Requirements and Deliverables and Technical Requirements Proposal Section IV: Narrative Responses Section VII: Pricing Model Vendor Presentations Proposed Work Plan
5.4.4 / p. 21 a. Is the expectation of “not to exceed $3M” to apply to proposals bidding all five components?
b. If pricing is for seven years, is inflation/COLA allowable past year 1?
a. The $3M (annual) is inclusive for all requirements (A-E)
b. All vendor price quotes
will be considered. We cannot be more definitive that what we have already specified in the RFP with is an NTE of $3M per year contingent upon federal and state funding and Governor and Executive Council approval.
A-1.7, A-1.8 / p. 26 Please confirm whether formative assessments—separate from summative and interim assessments—are required.
Formative assessments are not a required component for this RFP.
A-1.8 / p. 27 Please confirm, by subject area(s), whether the NHDOE is requesting an off-the-shelf product for the 2017-2018 school year, followed by a custom-developed assessment for subsequent years.
Vendor-developed solutions are acceptable. The subject areas included in this RFP are ELA, math and science.
RFP 2017-073 DOE New Hampshire Statewide Assessments
RFP Questions Received by 12:00 PM on April 6, 2017
Additional Questions from the Vendor Conference
C2 / p. 35 From page 35: “C-2 REQUIREMENTS
Vendors shall complete the requirements checklist (Table C-2 General Requirements Vendor Response Checklist). Table C-2 is included as an attachment to this RFP.”
Q: Is there a Table C-2 that was meant to be included as an attachment to this RFP?
Table C referred to on page 35 is the Excel Workbook attached to the RFP on the posting site. We apologize for the naming of the posted filed. It should read Table C.
Topic 2.5 / p. 48 Please confirm that the first administration of new field-test items is the 2018-2019 school year—after new ELA, math, and science standards are approved.
Section A-1.8 Field tested items may appear alongside operational items in all years including Year 1 (2017-2018). Scores must be reportable for accountability purposes in all years. *Science standards will remain as is through 2022.
Topic 17 / p. 66 “The vendor shall describe how the different types of scores it is proposing, individual scale scores and subscores, will be produced and verified. The vendor must include scores produced strictly on items which are computer-scorable and scores produced based on a combination of the computer-scored and hand-scored items. The limitations in interpretation of both of these scores must be discussed.”
Q: Please clarify or elaborate on what is meant here by subscores.
Subscore clarification using ELA as an example: Overall scale score for ELA is then broken into subscores for each claim achievement category: The Claims for English Language Arts are: •Reading •Writing •Speaking & Listening •Research/Inquiry
Topic 19.2 / p. 68 Please confirm that one or both of these similar processes will fulfill the requirement of “an independent real-time review of the equating process.”
The NH DOE may identify an independent reviewer to review the equating process, analyses, and results. The vendor(s) must support this effort by providing the consultant(s) with the
RFP 2017-073 DOE New Hampshire Statewide Assessments
RFP Questions Received by 12:00 PM on April 6, 2017
Additional Questions from the Vendor Conference
Third-party parallel equating
Third-party replication
If not, please describe the requirements of the process in more detail.
necessary data files and other materials in a timely manner during the equating process.
Appendix F/
pp. 100-103
a. Is Appendix F: (Pricing Worksheets) available in Excel format?
b. Should bidders budget for 4 years (through 2021) or 7 years (through 2024)?
An Excel file will be provided. Bidders should include a budget for the four years included on the RFP, through 2021. Bidders may include additional years or information if they choose to.
Appendix F/
pp. 100-103
a. What is the timeframe (# of years) vendors are to include in the Price category for Activities/Deliverables/ Milestones 1-35 in Table F-1?
b. Where should vendors capture pricing for activities not included in 1-35 (i.e. item development, committee meetings, scoring, reporting, analysis, etc.)?
Table F-1 should function as a Year 1 implementation and payment schedule. Vendors may add additional rows to the pricing worksheet for milestones not included in Table F-1. Vendors may include additional columns or tables for milestones and other items occurring subsequent to Year 1.
Appendix F/
pp. 100-103
a. What is the timeframe (# of years) vendors are to include in the Hours X Rate for Table F-2?
b. Please clarify “information is required by phase.”
Vendors should include Year 1 pricing in Table F- 2. Information is required for each implementation phase of the project.
Appendix F/
pp. 100-103
Please identify the specific years for Future Vendor Rates Worksheet Table F-3.
The information in Table F-3 is required
Pricing - General Would a proposal for all five components, with a bottom-line annual price of more than
Bidders should tailor proposals to meet the NTE of $3M.
RFP 2017-073 DOE New Hampshire Statewide Assessments
RFP Questions Received by 12:00 PM on April 6, 2017
Additional Questions from the Vendor Conference
$3 million, be automatically judged non-compliant and rejected? Or is any other scenario possible?
Pricing - NTE This RFP contains five components (A-E). Please clarify if the “not to exceed $3M” requirement is inclusive for ALL five components.
The $3M (annual) is inclusive for all requirements (A-E)
Page 42, Topic 2 Item Development, second paragraph:
While this RFP seeks an off-the-shelf solution, such a solution as well as a customized solution, may/will involve the creation of specific test items. This section dealing with Item Development applies to the development of items, whether used to tailor an off-the-shelf solution in response to this RFP or in a customized solution. Per NH RSA 193-C:3 III (d); “teachers will be involved in design and using the assessment system.” Much of the RFP seems intended to guide the custom development of a summative assessment. Considering the State is looking for an off-the-shelf solution, how does this involve New Hampshire educators in the item creation and on-going item review and release process if the State is seeking an out-of-the-box interim assessment?
NH Teachers must be involved in the on-going development of items and the item review.
Pages 39 – 95, Appendix D, Section D-1 through D-5
Are there requirements in the RFP that are intended for summative-only responses; since many of the requirements are directed at a summative assessment solution? If a
Yes there will be some tasks that can be omitted. For example: A concordance table is not necessary if only an interim solution is proposed.
RFP 2017-073 DOE New Hampshire Statewide Assessments
RFP Questions Received by 12:00 PM on April 6, 2017
Additional Questions from the Vendor Conference
vendor chose to submit only as an interim solution, are there any requirements that can be omitted because they are summative-only?
Page 92 – 95, Topic 45, Transition
Does the State intend to claim ownership of test items used in this solution, even if they were not developed for New Hampshire?
No the state does not intend to claim ownership of items not developed for New Hampshire.
Correction: Page 21 Section 5.4.4 Vendor proposed software solution cost will be allocated a maximum score of twenty-four (24) points. The State will consider both implementation costs and subsequent year license and maintenance costs, provided in Tables F-1: Activities/Deliverables/Milestones Pricing Worksheet, F-5: Software Licensing, Maintenance, and Support Pricing Worksheet and, if appropriate, F-6: Web Site Hosting, Maintenance, and Support Pricing Worksheet. Should read: F-4: Software Licensing, Maintenance, and Support Pricing Worksheet and, if appropriate, F-5: Web Site Hosting, Maintenance, and Support Pricing Worksheet Page 21 Section 5.4.4 “Vendor-proposed software solution cost” Vendor proposed software solution cost is inclusive of the total solution.
Topic 1 Test Design The summative assessments will use a common-matrix design to support a) the reporting of student-level overall science performance in terms of performance levels and scaled scores and b) the reporting of school- and district-level scores in a manner that reflects the depth and breadth of the academic standards. The word science should be omitted.
Update: Standards Revision and Alignment The NH academic standards for ELA, mathematics and science are entering a revision cycle. It is therefore imperative that vendors acknowledge and include in proposals a plan for assessment revision aligned to updated standards in 2018-2019.
RFP 2017-073 DOE New Hampshire Statewide Assessments
RFP Questions Received by 12:00 PM on April 6, 2017
Additional Questions from the Vendor Conference *The State Board of Education voted on April 6, 2017 to not begin the revision process for the NH
Academic Standards for Science until 2022.
*The State Board of Education will revisit the timeline for revision of the NH Academic Standards ELA and math at the May 11, 2017 Board meeting; this may alter the need for assessment re-alignment in
– The purpose of this voluntary pre-proposal vendor conference is to provide an informal forum for the potential proposers to ask questions and gain clarifications on the RFP requirements.
– Questions must be from potential proposers. Panel responses are considered informal and nonbinding.
– Proposers requiring a formal answer to their questions, or request for clarification or change must submit their questions in writing in accordance with Section 4.2 of the RFP.
– Subcontractors and vendors with questions shall submit through a potential proposer and not directly to NH DOE.
– Assessment system and associated support services for a computer-
based summative (with paper-based option) and interim assessment system aligned with New Hampshire academic standards in English Language Arts (ELA) and mathematics in grades 3-8 and science in grades 5, 8 and 11.
– Reporting capabilities to allow easy and timely access to results at various reporting levels from statewide down to the individual student/parent.
– ELA must include a machine-scored writing component.
– Contracts resulting from the RFP will require Governor and Council approval. – The NH DOE will post vendors names and rank or score on the website for
each respective vendor at least five days prior to submission of the Governor and Council contract package to the Department of Administrative Services.
– Confidential information should be clearly identified. Section 4.10
– Vendors may submit a written request to the DOE regarding the selection
process within 5 business days of bid results.
– The DOE will respond within 5 business days. No hearing will be held as part of the review. The outcome of the agency’s review will not be subject to appeal.
– The NH DOE Commissioner may waive the RFP guidelines in the event of an emergency or to prevent loss of funds subject to recapture.
– This RFP does not commit the State to award a contract.
– The State reserves the right, at its sole discretion, to reject any and all proposals, or any portions thereof.
– The State reserves the right to cancel this RFP and to solicit new proposals under a new acquisition process; or cancel this entire RFP or individual components at any time, without penalty.
Section 4.13
– In no event shall the State be either responsible for or held liable for any costs incurred by a vendor in the preparation of or in connection with a proposal, or for work performed prior to the effective date of a resulting contract.
– Cover Page – Transmittal Form Letter – Table of Contents – Section I: Executive Summary – Section II: Glossary of Terms and Abbreviations – Section III: Responses to Requirements and Deliverables – Section IV: Narrative Responses – Section V: Corporate Qualifications – Section VI: Qualifications of Key Vendor Staff – Section VII: Pricing Model – Section VIII: Copy of the RFP and any signed Addendum (a) - required in
• 9 points for A & B (summative); 5 points for C & D (interim) 21 points – Corp. Overview & Project Management 21 points – Project Execution and Ongoing Operations 24 points – Pricing Model
• 8 points for A & B (summative); 4 points for C & D (interim) 6 points – Overall Quality of Proposal 100 points – Total Possible Score
– The State will consider both implementation costs and subsequent year license and maintenance costs, provided in Tables F-1: Activities/Deliverables/Milestones Pricing Worksheet, F-5: Software Licensing, Maintenance, and Support Pricing Worksheet and, if appropriate, F-6: Web Site Hosting, Maintenance, and Support Pricing Worksheet.
Should read:
– F-4: Software Licensing, Maintenance, and Support Pricing Worksheet and, if appropriate, F-5: Web Site Hosting, Maintenance, and Support Pricing Worksheet
Clarification:
– “Vendor-proposed software solution cost”
– Vendor proposed software solution cost is inclusive of the total solution.
