OpenKIM Special-Purpose Tests and Verification Checks

This schema defines the manner in which Tests and Verification Checks (VCs) in openkim.org specify the types of Portable Models (PMs) and Simulator Models (SMs) with which they are compatible. The notion of SM subclasses is defined, which allows a Test or VC to provide specialized input for specific types of SMs. Tests and VCs that support SM subclasses are called "Special Purpose Tests" and "Special Purpose VCs."

The schema includes required keys in kimspec.edn for Tests, VCs, and SMs, and a definition of the matching logic for the OpenKIM Processing Pipeline (PP).

  1. SMs are divided into two categories: (1) Standard SMs that only require information about an atomic configuration available through the KIM API PM standard, and (2) Special-Purpose SMs that require additional information.

    1. The category of an SM is defined by a required run-compatibility key in its kimspec.edn file which contains a single string.

      • For a Standard SM, the run-compatibility key is set to portable-models.

      • For a Special-Purpose SM, the run-compatibility key is set to special-purpose-models.

    2. Note that a Standard SM may be able to receive additional information not available through the KIM API PM standard, but if this information is not provided the SM can still work. (An example is an SM that uses charges if provided, but otherwise uses default or internally computed values.)

    3. For the discussion below, we define "Standard Models" to be the set of all PMs and Standard SMs.

  2. Tests and VCs are divided into two categories: (1) Standard Tests/VCs that only work with Standard Models, and (2) Special-Purpose Tests/VCs that work with a subclass of Special-Purpose SMs.

    1. The category of a Test/VC is defined by a required matching-models key in its kimspec.edn file containing an array of strings. This key defines the set of models with which the Test/VC is compatible.

      • For a Standard Test/VC, the matching-models key can only contain ["standard-models"].

      • For a Special-Purpose Test/VC, the matching-models key contains a list of Special-Purpose SM types forming a subclass that the Test/VC supports.

    2. A subclass is a set of Special-Purpose SMs that do not need to be distinguished in terms of the input passed from a Test to its Test Driver (TD) or for the operation of a VC or standalone Test. An example is a set of bonded force field SMs that use the same input format, or a set of SMs that require charges that are specified in the same format.

    3. A subclass is specified as an array of strings each of which represents an individual SM type or a set of related SM types, e.g. ["iff/cvff", "iff/pcff"] or ["iff/"].

      • A hierarchy notation is used for SM type names. Names that do not end with a slash refers to a single SM type (e.g. iff refers to potentials of type iff, or iff/cvff refers to potentials of type iff/cvff). Names that end with a slash refer to all SM types within a category (e.g. iff/ refers to all potentials below iff/ in the hierarchy (but not including iff), such as iff/cvff, iff/pcff, iff/cvff/long, etc., and further subcategories if they exist.

      • The type of an SM is specified by the required simulator-potential key in its kimspec.edn file.

      • A subclass cannot contain standard-models as a member.

      • A subclass is associated with a single Simulator as specified by the simulator-name key in the kimspec.edn file of the Test/VC.

    4. A subclass specification is local to a particular Special-Purpose Test or VC. The OpenKIM system does not define or track subclasses as an independent concept.

  3. A TD may support Standard Models as well as multiple subclasses of Special-Purpose SMs. However each Test associated with the TD is limited to either Standard Models or a single subclass. (This is done to keep the parameter files associated with an individual Test from becoming unwieldy due to the need to support the requirements of different types of models.) For example if the TD supports both Standard Models and a subclass requiring charges, then the TD would need to have separate Tests for these cases. The Test parameter file would contain information identifying it, for example a key set to either standard or charge, where for charge the parameter file would include additional charge information not provided by the KIM API PM standard.

  4. It is possible for an SM to be a Standard Model and/or belong to more than one subclass. This means that an SM could be run by more than one Test associated with a TD. For example, a Standard SM may also use charges if provided as noted in item 1b. When run by a Standard Test/VC such a model would use default values for charges, but with a Special-Purpose Test/VC it would initialize the charges to provided values. This could lead to some confusion, but it also provides valuable information on the predictions of the SM under different modes of operation. Thus there can be multiple results for a single model (and species) for the same TD.

  5. The PP matching logic for Standard and Special-Purpose SMs consistent with the above points is defined below. (Note that in the following "compatible" means passing additional PP matching criteria such as species, KIM API version, and simulator.)

    A compatible Test/VC and Model are considered to match according to the following table:

    Test/VC matching-models list A PM SM
    A=["standard-models"] yes run-compatibility == "portable-models"
    A=[...] no simulator-potential in A