Tutorial on Developing KIM tests using ASE

Test | Example calculation


Here, we describe a basic test using the Atomic Simulation Environment (ASE) named ASECohesiveEnergyFromQueryExample_fcc_Ar__TE_102111117114_003. This test is designed to (1) retrieve the equilibrium fcc lattice constant for argon for whichever model it is being run against by querying the OpenKIM repository and (2) calculate the cohesive energy corresponding to this lattice constant for that model.

List of files

All KIM tests are required to have the following files:

  • Makefile - trivial makefile (does nothing since no compilation is needed)
  • kimspec.edn - metadata file that describes the test
  • pipeline.stdin.tpl - Jinja-formatted template file that is processed by the KIM processing pipeline and piped to the test on stdin
  • runner - the main executable used to run the test

In addition to the required files above, this test also contains:

  • LICENSE.CDDL - a copy of the CDDL license
  • dependencies.edn - specifies other tests whose results this test uses


This is the main executable of the test. In this case, it so happens to be a python script that takes the name of a KIM model as input, performs a query to the OpenKIM repository to retrieve the fcc lattice constant for argon predicted by that model, and computes the cohesive energy at that lattice geometry.

NOTE Every KIM test and test driver is required to have a primary executable named runner, although this executable is free to call other executables, etc.

#!/usr/bin/env python3
ASE cohesive energy example test with dependencies
from ase.build import bulk
from ase.calculators.kim import KIM
from kim_query import get_lattice_constant_cubic
from kim_property import kim_property_create, kim_property_modify, kim_property_dump

# Grab the model name from standard input stream (stdin)
model = input("Enter a KIM model name:\n")

# Query for the fcc lattice constant
lattice_constant = get_lattice_constant_cubic([model], ["fcc"], ["Ar"], ["angstrom"])[0]

# Set up a primitive FCC Ar unit cell with periodic boundary conditions
atoms = bulk('Ar', 'fcc', a=lattice_constant)

# Initialize instance of the KIM ASE calculator for the given model and attach to atoms
# object
calc = KIM(model)

# Compute cohesive energy (and make positive to conform to sign convention in
# property definition 'cohesive-potential-energy-cubic-crystal')
ecohesive = -atoms.get_potential_energy()

# Echo to output
print(f"Cohesive energy = {ecohesive} eV/atom")

# Create a property instance
prop_instance = kim_property_create(1, "cohesive-potential-energy-cubic-crystal")

# Set all the key-value pairs for this property instance
prop_instance = kim_property_modify(prop_instance, 1,
                             "key", "short-name", "source-value", 1, "fcc",
                             "key", "species", "source-value", 1, "Ar",
                             "key", "a", "source-value", lattice_constant,
                                         "source-unit", "angstrom",
                             "key", "basis-atom-coordinates", "source-value", "1", "1:3", 0.0, 0.0, 0.0,
                             "key", "basis-atom-coordinates", "source-value", "2", "1:3", 0.0, 0.5, 0.5,
                             "key", "basis-atom-coordinates", "source-value", "3", "1:3", 0.5, 0.0, 0.5,
                             "key", "basis-atom-coordinates", "source-value", "4", "1:3", 0.5, 0.5, 0.0,
                             "key", "space-group", "source-value", "Fm-3m",
                             "key", "cohesive-potential-energy", "source-value", ecohesive,
                                                                 "source-unit", "eV")

# Dump the results to results.edn file in `output` subdir
kim_property_dump(prop_instance, "output/results.edn")

The test begins by grabbing a KIM model name from the standard input (stdin) stream (cf. the section for pipeline.stdin.tpl below). Next, the test performs a query to retrieve the fcc argon lattice constant predicted by this model. This is done using the get_lattice_constant_cubic function from the kim-query package, which provides a convenient way to look up already-computed test results from the OpenKIM repository.1 With the arguments provided, this specific function looks up the lattice constant computed by the latest version of a test named LatticeConstantCubicEnergy_fcc_Ar__TE_206669103745 when it was run against the current model (the latest version is currently LatticeConstantCubicEnergy_fcc_Ar__TE_206669103745_007). Using the queried lattice constant, a primitive (one-atom) fcc Ar unit cell with periodic boundary conditions is constructed. Next, an instance of the KIM ASE calculator is initialized using the inputted model name and is attached to the ASE Atoms object before its get_potential_energy method is called.

In order for the results produced by a test to be useful, the relevant physical information must be recorded in a specific format whose pieces have well-defined meaning. To this end, every test is required to report instances of KIM property definitions. While this can be done in various ways, the kim-property python package has been created to assist in this task. In this test, once the energy has been computed, the kim_property_create function is used to initialize a property instance corresponding to the cohesive-potential-energy-cubic-crystal property definition.2 Next, the kim_property_modify function is used to fill in the property instance with the relevant quantities including the atomic species, specific cubic lattice type, the cohesive energy computed, etc. Note that the original energy obtained using the ASE calculator had its sign reversed. Each property definition generally has its own conventions for how physical information is reported; the property definition reported by this test follows the convention that stable lattices have positive cohesive energy.

Finally, the property instance is written to a file named results.edn in a subdirectory named output using the kim_property_dump function. There are two conventions of the KIM processing pipeline at work here. First, in order to prevent a KIM test from inadvertently including artifacts and other unnecessary files in the test result it generates, all files that are to be retained as part of the test result must be explicitly written or copied to a subdirectory named output (which the pipeline automatically creates for you before your test is run). Second, the results of a test must be written into a file specifically named results.edn.


This is a Jinja2-formatted template file that the OpenKIM processing pipeline will fill in and provide to the test on stdin when it is run against a model.


The pipeline's custom Jinja environment features the notation @<…>@ for enclosing python code you wish to execute, while @[…]@ denotes code blocks and @#…#@ denotes comments. Many standard Python functions can thus be used inside of @<…>@ delimiters, as well as a few special functions and keywords specific to KIM. Here, the special keyword MODELNAME will be replaced by the name of the model that the pipeline is currently running the test against.

NOTE Every KIM test requires a file named pipeline.stdin.tpl which, if nothing else, is used to input the name of the KIM model currently being run against.


This file is used internally by the pipeline to ensure that tests are run in the appropriate order. If your test performs queries to retrieve the results of other tests, they should be recorded here (if your test does perform any such queries, this file should be omitted). At the moment, determining the specific dependencies of your test must currently be done manually. In this case, if we consult the documentation for the get_lattice_constant_cubic function used to perform the query, we can see that it queries for results produced by tests that use the test driver family with code TD_475411767977. On the page for the latest version of this driver, LatticeConstantCubicEnergy__TD_475411767977_007, we can see that its test that computes the fcc Ar lattice constant is LatticeConstantCubicEnergy_fcc_Ar__TE_206669103745_007. Because a test is only allowed to query for results produced by the latest versions of other tests (as of the time it is run), the 3-digit version extension is omitted when it's listed in dependencies.edn. In fact, the human-readable prefix can be omitted as well, as in the case of the current test:

[ "TE_206669103745" ]

Tests that have multiple dependencies should separate them by spaces, in accorance with the EDN format (see About the EDN data format).


This is a metadata file that describes the test.

  "title" "ASE cohesive energy example test v003"
  "extended-id" "ASECohesiveEnergyFromQueryExample_fcc_Ar__TE_102111117114_003"
  "species" [
  "properties" [
  "author" "Matt Bierbaum"
  "description" "Tutorial test using the Atomic Simulation Environment (ASE).  Given a model for Ar, it performs a query to retrieve the FCC lattice constant and computes the cohesive energy.\n\n\nHISTORY:\n\nChanges in version 003:\n* Use kim-query package to query instead of putting it in pipeline.stdin.tpl\n* Use kim-property package to write property instance"
  "contributor-id" "8f8225b4-8b9c-439d-879d-45ee35db5757"
  "maintainer-id" "4d62befd-21c4-42b8-a472-86132e6591f3"
  "publication-year" "2020"
  "kim-api-version" "2.0"
  "executables" [
  "domain" "openkim.org"

NOTE When working on a test inside the KIM Developer Platform, you must create a kimspec.edn file for your test that lists its "extended-id" and, if the test uses a test driver, the "test-driver" field. It must also list the "species" supported by your test (in the form of an EDN array, e.g. [ "Ni" "Al" ]) and the minimum KIM API version your test is compatible with. The other fields aren't necessary until you're ready to upload your test (and can actually be filled in automatically for you as you complete the upload forms).


In this example, the Makefile does nothing since no compilation is needed

	@echo "Nothing to make"

	@echo "Nothing to clean"

Example Calculation

On the KIM Developer Platform, this test can be installed using the kimitems utility:

kimitems install ASECohesiveEnergyFromQueryExample_fcc_Ar__TE_102111117114_003

which will download a tarball of the test from openkim.org, decompress it into ~/tests/, and attempt to build it.

Suppose we now wish to run this test against LJ_Shifted_Bernardes_1958MedCutoff_Ar__MO_126566794224_004 model. After installing the model via

kimitems install LJ_Shifted_Bernardes_1958MedCutoff_Ar__MO_126566794224_004

we can run this specific test–model pair by issuing

pipeline-run-pair ASECohesiveEnergyFromQueryExample_fcc_Ar__TE_102111117114_003 LJ_Shifted_Bernardes_1958MedCutoff_Ar__MO_126566794224_004

This will result in output like this:

+ Running pair (ASECohesiveEnergyFromQueryExample_fcc_Ar__TE_102111117114_003, LJ_Shifted_Bernardes_1958MedCutoff_Ar__MO_126566794224_004)

  Pair produced Test Result TE_102111117114_003-and-MO_126566794224_004-1583860662-tr in 0.20717191696166992 seconds

This indicates that our test–model pair produced a test result (as opposed to an error), and that the test result was placed in ~/test-results/TE_102111117114_003-and-MO_126566794224_004-1583860662-tr. Let's go to this folder and inspect its contents:

├── kim.log - log file created by the KIM API
├── kimspec.edn - metadata for the test result
├── pipeline.stderr - stderr output from the test
├── pipeline.stdin - final stdin that was fed to the test
├── pipeline.stdout - stdout output from the test
├── pipelinespec.edn - profiling information, including runtime
└── results.edn - final property instance results file output by the test

If we take a look at the pipeline.stdin file, we can see that the MODELNAME in the pipeline.stdin.tpl input template was successfully rendered to the name of the model we ran the test against:


and from results.edn, we can see that the cohesive energy computed by the test was approximately 0.0865 eV:

    "property-id" "tag:staff@noreply.openkim.org,2014-04-15:property/cohesive-potential-energy-cubic-crystal"
    "instance-id" 1
    "short-name" {
        "source-value" [
    "species" {
        "source-value" [
    "a" {
        "source-value" 5.248509056866169
        "source-unit" "angstrom"
        "si-unit" "m"
        "si-value" 5.248509056866169e-10
    "basis-atom-coordinates" {
        "source-value" [
    "space-group" {
        "source-value" "Fm-3m"
    "cohesive-potential-energy" {
        "source-value" 0.08650550849505378
        "source-unit" "eV"
        "si-unit" "kg m^2 / s^2"
        "si-value" 1.38597103281191e-20


  • Don't waste time typing when you don't have to! Tab-completion is supported over the KIM Developer Platform command-line utility names, their options, and currently installed KIM tests, models, and verification checks, etc.

  • To run this test against all compatible models and simulator models currently installed in your KIM Developer Platform, you can issue pipeline-run-matches ASECohesiveEnergyFromQueryExample_fcc_Ar__TE_102111117114_003.

  • Although not so useful here, the -v/--verbose flag to the pipeline-run-* utilities prints the stdout and stderr streams of the test to the terminal while it is running.

  1. For a detailed listing of query functions with examples, see here.

  2. To see a list of all current KIM property definitions, click here