Testing lang

The following is a port of the previously proposed string format. YAML was selected for both ubiquitous language support and support for inline comments. The short-hand string format will likely still be supported via a python tool that converts it to this YAML format.

YAML fields

title (required)

summary (optional)

test_suite (required) “chain_test” defines this test suite

fork (required) versioning

test_cases (required) is a set of key value pairs of the form {test_name: test_case} where test_name is a short string name of the test and test_case is an object conforming the the “chain_test” test case standard defined below.

“chain_test” test case

config (required) represents the test configuration.

slots (required) represents what exactly happens in each slot. If a slot_number is missing, it is assumed that nothing happens during that slot. Slots should be processed in order of slot_number. A slot is made up of the following fields:

• slot_number (required) defines the slot number. The first slot after genesis (and thus the minimum slot_number that can be specified) is 1.
• new_block (optional) defines the new block to be proposed at that slot_number. If present, a new block is proposed during that slot with the corresponding id and parent. If new_block is left out, then no new block is proposed for that slot_number.
• attestations (optional) defines a list of attestations that occur during the slot_number. If left out or consisting of zero elements, then no attestations occur at that slot_number. An attestation is an object made up of the following
  • block (required) is a block id defined at or before the current slot_number to which the attestation attests.
  • validators (required) is a list of validator indices that are included in the attestion. This list is composed of indices to the committee(s) from the committee_slot of the attestation (not indices of the global validators set). In practice, these indices are from the shard_and_committee_for_slots for the current slot. If multiple committees are in a given slot, concatenate these together.
  • committee_slot (optional) defines which slot the list of validators are drawn from. If committee_slot is left blank, it is assumed to be the slot_number of the attestation in question.

results (optional) represents the expected results of the fork choice and finality gadget that are to be checked after applying the sequence of actions defined by slots. If left out or any of the results fields are not included, the test runner is expected to output whatever the local system calculates the results to be.

Sample YAML file

---

title: Sample Eth2.0 beacon chain test
summary: YAML conversion of sample test string format: https://notes.ethereum.org/6ozACHyKR9i86vfkt086Ow
test_suite: chain_test
fork: Tchoupitoulas

test_cases:
  test_case_1:
    config:
      validator_count: 100
      cycle_length: 8
      shard_count: 32
      min_committee_size: 8

    slots:
      # "slot_number" has a minimum of 1
      - slot_number: 1
        new_block:
          id: A
          # "*" is used for the genesis block
          parent: "*"
        attestations:
          - block: A
            # the following is a shorthand string for [0, 1, 2, 3, 4, 5]
            validators: "0-5"
      - slot_number: 2
        new_block:
          id: B
          parent: A
        attestations:
          - block: B
            validators: [0, 1, 2, 3, 4, 5]
      - slot_number: 3
        new_block:
          id: C
          parent: A
        attestations:
          # attestation "committee_slot" defaults to the slot during which the attestation occurs
          - block: C
            validators: "2-7"
          # default "committee_slot" can be directly overridden
          - block: C
            committee_slot: 2
            validators: [6, 7]
      - slot_number: 4
        new_block:
          id: D
          parent: C
        attestations:
          - block: D
            validators: "1-4"
      # slots can be skipped entirely (5 in this case)
      - slot_number: 6
        new_block:
          id: E
          parent: D
        attestations:
          - block: E
            validators: "0-4"
          - block: B
            validators: [5, 6, 7]

    results:
      head: E
      last_justified_block: "*"
      last_finalized_block: "*"