About the Testing Qualifiction Process

General policy guidelines for financial management systems set forth in OMB Circular A-127 requires agencies to establish and maintain a single, integrated financial management system. A-127 further requires agencies replacing financial system software to acquire an “off-the-shelf” product that has been certified by JFMIP.

JFMIP periodically tests commercial financial management software products in order to “certify” them for federal acquisition. This testing process is designed to ensure that available products comply with all mandatory core financial management systems requirements, also published by JFMIP. The objective of this program is to enable agencies to acquire new financial management systems with the assurance selected products can handle common federal accounting, reporting, and electronic business transactions out of the box. The testing program is a multi-year cycle of activities that starts with agency outreach to identify requirements. This process ends in a formal system test that every commercial vendor must pass. Test results are made available to agencies to help them with product acquisition and upgrade decisions.

Requirements Identification

Federal financial managers rely on JFMIP published requirements as the official reference describing the essential characteristics of their automated systems. Auditors of federal financial management systems rely on JFMIP requirements as their official reference for required systems functionality. Vendors rely on JFMIP requirements to help them understand agency-processing needs, enabling them to develop improved system solutions.

While qualification testing is currently limited to mandatory core FMS requirements, JFMIP’s requirements identification efforts are far more expansive covering core plus financial system impacts from:

• Human resources and payroll
• Inventory
• Travel acquisition
• Seized property and forfeited assets
• Direct and guaranteed loans
• Benefit payments grants management
• Property management
• Revenue management
• Managerial cost accounting feeder systems

The basic process followed by JFMIP to update a requirements document is: 1) organize a project; 2) assemble a draft set of updated requirements based on new legislation, oversight agency guidance and emerging business practices: 3) distribute a draft to stakeholders for comment; and 4) publish resulting requirements in a final form as approved by the JFMIP Steering Committee.

Test Development

There are two types of tests - full and incremental. Full tests are performed on all qualified products once every three years. Full tests cover all mandatory core requirements in force at the time the test is being developed. JFMIP reserves the right to conduct incremental tests annually. Incremental tests are designed to test package compliance with emerging financial system requirements as announced by Treasury, OMB or other accounting oversight body from time to time. Whether incremental or full, a qualification test is developed much like a system—using specifications, development, quality assurance, and rollout.

For the full test version, JFMIP will translate approximately 600 mandatory functional requirements into a series of business cases, test steps, and posting transactions that are suitable for execution in a wide variety of operating environments. This effort requires considerable expertise including:

• Functional expertise—USSGL, funding, payables, receivables, cost, and reporting.
• Systems expertise—Architectures, platforms, interoperability, web access, workflow, security, and operations.
• Test expertise—Scripting, automated execution, configuration, benchmarking, and capacity planning.

The goal is to build a resource that will provide the government with an unimpeachable basis for determining whether a package is qualified for federal government use. In writing actual test steps, JFMIP translates mandatory requirements into a set of system inputs, posting rules, and expected outputs suitable for execution in a wide variety of package architectures.

The test produced in 2003 contained more than 700 test steps and 200,000 data points. To manage this complexity, JFMIP used a custom software tool called the JFMIP Computer-Aided Test Tool. JCATT stores test cases, steps, and transactions and is used to generate a complete set of expected results including financial statements, transaction registers, trial balances, and Treasury e-commerce files.

Product Testing

Qualification testing starts with a vendor application that identifies the subject package and participating staff. The process typically ends with the issuance of a certificate of qualification good for approximately 3 years.

Specific tasks undertaken by JFMIP during a test include 1) vendor administration to schedule a test and provide technical support, 2) test execution to enter steps and deal with execution problems, 3) validation of collected outputs, and 4) certification/publishing of results.

In taking a test, vendors must configure their packages to JFMIP specifications, enter the data, and generate the results. While actual execution only lasts 2-weeks, vendors can take 3 months to complete this process. During this time, vendors must provide computing facilities and the software subject to qualification. Test results for all packages that completed in the 2003 cycle can be found on this web site.

Agency Outreach

Agencies are integral to the success of JFMIP qualification testing process. Common agency needs drive the requirements identification process on the front end. Agency representatives help establish test development priorities and are primary audience for published test results.

To help agencies stay engaged in the testing process, JFMIP actively solicits their input using a variety of special study groups, topical workshops, open houses, surveys and key personnel interviews. The resulting proceedings, reports, and other requirements testing-related materials are regularly published by JFMIP.

JFMIP uses web-based applications such as this Knowledgebase to disseminate program information and engage agencies in various sponsored activities.

		FSIO is committed to Section 508 compliance and accessibility of websites. We welcome you to contact us with any ideas or comments that may help us improve upon the accessibility and usability of our website. View the FSIO Privacy Statement
		USAGov: The U.S. Government Homepage

FINANCIAL SYSTEMS INTEGRATION OFFICE
1800 F St. N.W., Rm 1234
Washington DC 20405