The targets in src/testdir/Makefile indicate that the Vim code runs tests as multi-step tests: first the vim executable processes a script and produces an output file, then in a second step the output file is compared with a reference file and if these files do not differ, the test is successful. Temporary files are then removed in a third step. This can probably not be fitted into a single add_test command in a portable way since add_test can only execute one command. One solution would be to define the test steps in a Python script and to execute the Python script with some arguments. The alternative we will present here, which is also cross-platform, is to define the test steps in a separate CMake script and to execute this script from add_test.  We will define the test steps in src/testdir/test.cmake:

function(execute_test _vim_executable _working_dir _test_script)
  # generates test.out
  execute_process(
    COMMAND ${_vim_executable} -f -u unix.vim -U NONE --noplugin --not-a-term -s dotest.in ${_test_script}.in
    WORKING_DIRECTORY ${_working_dir}
    )

  # compares test*.ok and test.out
  execute_process(
    COMMAND ${CMAKE_COMMAND} -E compare_files ${_test_script}.ok test.out
    WORKING_DIRECTORY ${_working_dir}
    RESULT_VARIABLE files_differ
    OUTPUT_QUIET
    ERROR_QUIET
    )

  # removes leftovers
  file(REMOVE ${_working_dir}/Xdotest)

  # we let the test fail if the files differ
  if(files_differ)
    message(SEND_ERROR "test ${_test_script} failed")
  endif()
endfunction()

execute_test(${VIM_EXECUTABLE} ${WORKING_DIR} ${TEST_SCRIPT})

Again, we choose a function over a macro to make sure variables do not escape the function scope. We will process this script, which will call the execute_test function. However, we have to make sure that ${VIM_EXECUTABLE}, ${WORKING_DIR}, and ${TEST_SCRIPT} are defined from outside. These are defined in src/testdir/CMakeLists.txt:

add_test(
  NAME
    test1
  COMMAND
    ${CMAKE_COMMAND} -D VIM_EXECUTABLE=$<TARGET_FILE:vim>
                     -D WORKING_DIR=${CMAKE_CURRENT_LIST_DIR}
                     -D TEST_SCRIPT=test1
                     -P ${CMAKE_CURRENT_LIST_DIR}/test.cmake
  WORKING_DIRECTORY
    ${PROJECT_BINARY_DIR}
  )

The Vim project has many tests but in this example, we have ported only one (test1) as a proof of concept.