tests/functional: Convert the acpi-bits test into a standalone test

Mostly a straight-forward conversion. Looks like we can simply drop
the avocado datadrainer stuff when not using the avocado framework
anymore.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-30-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

MAINTAINERS

@@ -2074,8 +2074,8 @@ ACPI/AVOCADO/BIOSBITS
 M: Ani Sinha <anisinha@redhat.com>
 M: Michael S. Tsirkin <mst@redhat.com>
 S: Supported
-F: tests/avocado/acpi-bits/*
+F: tests/functional/acpi-bits/*
-F: tests/avocado/acpi-bits.py
+F: tests/functional/test_acpi_bits.py
 F: docs/devel/acpi-bits.rst
 
 ACPI/HEST/GHES

docs/devel/acpi-bits.rst

@@ -1,6 +1,6 @@
-=============================================================================
+==================================
-ACPI/SMBIOS avocado tests using biosbits
+ACPI/SMBIOS testing using biosbits
-=============================================================================
+==================================
 ************
 Introduction
 ************
@@ -35,7 +35,7 @@ for developing biosbits and its real life uses can be found in [#a]_ and [#b]_.
 For QEMU, we maintain a fork of bios bits in gitlab along with all the
 dependent submodules `here <https://gitlab.com/qemu-project/biosbits-bits>`__.
 This fork contains numerous fixes, a newer acpica and changes specific to
-running this avocado QEMU tests using bits. The author of this document
+running these functional QEMU tests using bits. The author of this document
 is the sole maintainer of the QEMU fork of bios bits repository. For more
 information, please see author's `FOSDEM talk on this bios-bits based test
 framework <https://fosdem.org/2024/schedule/event/fosdem-2024-2262-exercising-qemu-generated-acpi-smbios-tables-using-biosbits-from-within-a-guest-vm-/>`__.
@@ -44,12 +44,12 @@ framework <https://fosdem.org/2024/schedule/event/fosdem-2024-2262-exercising-qe
 Description of the test framework
 *********************************
 
-Under the directory ``tests/avocado/``, ``acpi-bits.py`` is a QEMU avocado
+Under the directory ``tests/functional/``, ``test_acpi_bits.py`` is a QEMU
-test that drives all this.
+functional test that drives all this.
 
 A brief description of the various test files follows.
 
-Under ``tests/avocado/`` as the root we have:
+Under ``tests/functional/`` as the root we have:
 
 ::
 
@@ -60,12 +60,12 @@ Under ``tests/avocado/`` as the root we have:
    │   ├── smbios.py2
    │   ├── testacpi.py2
    │   └── testcpuid.py2
-   ├── acpi-bits.py
+   ├── test_acpi_bits.py
 
-* ``tests/avocado``:
+* ``tests/functional``:
 
-   ``acpi-bits.py``:
+   ``test_acpi_bits.py``:
-   This is the main python avocado test script that generates a
+   This is the main python functional test script that generates a
    biosbits iso. It then spawns a QEMU VM with it, collects the log and reports
    test failures. This is the script one would be interested in if they wanted
    to add or change some component of the log parsing, add a new command line
@@ -79,35 +79,22 @@ Under ``tests/avocado/`` as the root we have:
    you to inspect and run the specific commands manually.
 
    In order to run this test, please perform the following steps from the QEMU
-   build directory:
+   build directory (assuming that the sources are in ".."):
    ::
 
-     $ make check-venv (needed only the first time to create the venv)
+     $ export PYTHONPATH=../python:../tests/functional
-     $ ./pyvenv/bin/avocado run -t acpi tests/avocado
+     $ export QEMU_TEST_QEMU_BINARY=$PWD/qemu-system-x86_64
+     $ python3 ../tests/functional/test_acpi_bits.py
 
-   The above will run all acpi avocado tests including this one.
+   The above will run all acpi-bits functional tests (producing output in
-   In order to run the individual tests, perform the following:
+   tap format).
-   ::
 
-     $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py --tap -
+   You can inspect the log files in tests/functional/x86_64/test_acpi_bits.*/
+   for more information about the run or in order to diagnoze issues.
+   If you pass V=1 in the environment, more diagnostic logs will be put into
+   the test log.
 
-   The above will produce output in tap format. You can omit "--tap -" in the
+* ``tests/functional/acpi-bits/bits-config``:
-   end and it will produce output like the following:
-   ::
-
-      $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py
-      Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
-      JOB ID     : eab225724da7b64c012c65705dc2fa14ab1defef
-      JOB LOG    : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log
-      (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (33.09 s)
-      RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
-      JOB TIME   : 39.22 s
-
-   You can inspect the log file for more information about the run or in order
-   to diagnoze issues. If you pass V=1 in the environment, more diagnostic logs
-   would be found in the test log.
-
-* ``tests/avocado/acpi-bits/bits-config``:
 
    This location contains biosbits configuration files that determine how the
    software runs the tests.
@@ -117,7 +104,7 @@ Under ``tests/avocado/`` as the root we have:
    or actions are performed by bits. The description of the config options are
    provided in the file itself.
 
-* ``tests/avocado/acpi-bits/bits-tests``:
+* ``tests/functional/acpi-bits/bits-tests``:
 
    This directory contains biosbits python based tests that are run from within
    the biosbits environment in the spawned VM. New additions of test cases can
@@ -155,7 +142,8 @@ Under ``tests/avocado/`` as the root we have:
    (a) They are python2.7 based scripts and not python 3 scripts.
    (b) They are run from within the bios bits VM and is not subjected to QEMU
        build/test python script maintenance and dependency resolutions.
-   (c) They need not be loaded by avocado framework when running tests.
+   (c) They need not be loaded by the test framework by accident when running
+       tests.
 
 
 Author: Ani Sinha <anisinha@redhat.com>

tests/functional/meson.build

@@ -11,6 +11,7 @@ endif
 
 # Timeouts for individual tests that can be slow e.g. with debugging enabled
 test_timeouts = {
+  'acpi_bits' : 240,
   'netdev_ethtool' : 180,
   'ppc_40p' : 240,
   'ppc64_hv' : 1000,
@@ -96,6 +97,7 @@ tests_x86_64_system_quick = [
 ]
 
 tests_x86_64_system_thorough = [
+  'acpi_bits',
   'netdev_ethtool',
   'virtio_gpu',
 ]

tests/functional/test_acpi_bits.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# group: rw quick
+#
 # Exercise QEMU generated ACPI/SMBIOS tables using biosbits,
 # https://biosbits.org/
 #
@@ -24,7 +24,7 @@
 # pylint: disable=consider-using-f-string
 
 """
-This is QEMU ACPI/SMBIOS avocado tests using biosbits.
+This is QEMU ACPI/SMBIOS functional tests using biosbits.
 Biosbits is available originally at https://biosbits.org/.
 This test uses a fork of the upstream bits and has numerous fixes
 including an upgraded acpica. The fork is located here:
@@ -41,15 +41,16 @@ import tarfile
 import tempfile
 import time
 import zipfile
+
+from pathlib import Path
 from typing import (
     List,
     Optional,
     Sequence,
 )
 from qemu.machine import QEMUMachine
-from avocado import skipIf
+from unittest import skipIf
-from avocado.utils import datadrainer as drainer
+from qemu_test import QemuBaseTest, Asset
-from avocado_qemu import QemuBaseTest
 
 deps = ["xorriso", "mformat"] # dependent tools needed in the test setup/box.
 supported_platforms = ['x86_64'] # supported test platforms.
@@ -129,34 +130,32 @@ class QEMUBitsMachine(QEMUMachine): # pylint: disable=too-few-public-methods
 class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
     """
     ACPI and SMBIOS tests using biosbits.
-
-    :avocado: tags=arch:x86_64
-    :avocado: tags=acpi
-
     """
     # in slower systems the test can take as long as 3 minutes to complete.
     timeout = BITS_TIMEOUT
 
+    # following are some standard configuration constants
+    # gitlab CI does shallow clones of depth 20
+    BITS_INTERNAL_VER = 2020
+    # commit hash must match the artifact tag below
+    BITS_COMMIT_HASH = 'c7920d2b'
+    # this is the latest bits release as of today.
+    BITS_TAG = "qemu-bits-10262023"
+
+    ASSET_BITS = Asset(("https://gitlab.com/qemu-project/"
+                        "biosbits-bits/-/jobs/artifacts/%s/"
+                        "download?job=qemu-bits-build" % BITS_TAG),
+                       '1b8dd612c6831a6b491716a77acc486666aaa867051cdc34f7ce169c2e25f487')
+
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
         self._vm = None
         self._workDir = None
         self._baseDir = None
 
-        # following are some standard configuration constants
-        self._bitsInternalVer = 2020 # gitlab CI does shallow clones of depth 20
-        self._bitsCommitHash = 'c7920d2b' # commit hash must match
-                                          # the artifact tag below
-        self._bitsTag = "qemu-bits-10262023" # this is the latest bits
-                                             # release as of today.
-        self._bitsArtSHA1Hash = 'b22cdfcfc7453875297d06d626f5474ee36a343f'
-        self._bitsArtURL = ("https://gitlab.com/qemu-project/"
-                            "biosbits-bits/-/jobs/artifacts/%s/"
-                            "download?job=qemu-bits-build" %self._bitsTag)
         self._debugcon_addr = '0x403'
         self._debugcon_log = 'debugcon-log.txt'
-        logging.basicConfig(level=logging.INFO)
+        self.logger = self.log
-        self.logger = logging.getLogger('acpi-bits')
 
     def _print_log(self, log):
         self.logger.info('\nlogs from biosbits follows:')
@@ -171,7 +170,7 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
         bits_config_dir = os.path.join(self._baseDir, 'acpi-bits',
                                        'bits-config')
         target_config_dir = os.path.join(self._workDir,
-                                         'bits-%d' %self._bitsInternalVer,
+                                         'bits-%d' %self.BITS_INTERNAL_VER,
                                          'boot')
         self.assertTrue(os.path.exists(bits_config_dir))
         self.assertTrue(os.path.exists(target_config_dir))
@@ -188,7 +187,7 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
         bits_test_dir = os.path.join(self._baseDir, 'acpi-bits',
                                      'bits-tests')
         target_test_dir = os.path.join(self._workDir,
-                                       'bits-%d' %self._bitsInternalVer,
+                                       'bits-%d' %self.BITS_INTERNAL_VER,
                                        'boot', 'python')
 
         self.assertTrue(os.path.exists(bits_test_dir))
@@ -248,9 +247,9 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
             test scripts
         """
         bits_dir = os.path.join(self._workDir,
-                                'bits-%d' %self._bitsInternalVer)
+                                'bits-%d' %self.BITS_INTERNAL_VER)
         iso_file = os.path.join(self._workDir,
-                                'bits-%d.iso' %self._bitsInternalVer)
+                                'bits-%d.iso' %self.BITS_INTERNAL_VER)
         mkrescue_script = os.path.join(self._workDir,
                                        'grub-inst-x86_64-efi', 'bin',
                                        'grub-mkrescue')
@@ -264,8 +263,12 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
 
         try:
             if os.getenv('V') or os.getenv('BITS_DEBUG'):
-                subprocess.check_call([mkrescue_script, '-o', iso_file,
+                proc = subprocess.run([mkrescue_script, '-o', iso_file,
-                                       bits_dir], stderr=subprocess.STDOUT)
+                                       bits_dir],
+                                      stdout=subprocess.PIPE,
+                                      stderr=subprocess.STDOUT,
+                                      check=True)
+                self.logger.info("grub-mkrescue output %s" % proc.stdout)
             else:
                 subprocess.check_call([mkrescue_script, '-o',
                                       iso_file, bits_dir],
@@ -282,8 +285,9 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
 
     def setUp(self): # pylint: disable=arguments-differ
         super().setUp('qemu-system-')
+        self.logger = self.log
 
-        self._baseDir = os.getenv('AVOCADO_TEST_BASEDIR')
+        self._baseDir = Path(__file__).parent
 
         # workdir could also be avocado's own workdir in self.workdir.
         # At present, I prefer to maintain my own temporary working
@@ -300,15 +304,14 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
             os.mkdir(prebuiltDir, mode=0o775)
 
         bits_zip_file = os.path.join(prebuiltDir, 'bits-%d-%s.zip'
-                                     %(self._bitsInternalVer,
+                                     %(self.BITS_INTERNAL_VER,
-                                       self._bitsCommitHash))
+                                       self.BITS_COMMIT_HASH))
         grub_tar_file = os.path.join(prebuiltDir,
                                      'bits-%d-%s-grub.tar.gz'
-                                     %(self._bitsInternalVer,
+                                     %(self.BITS_INTERNAL_VER,
-                                       self._bitsCommitHash))
+                                       self.BITS_COMMIT_HASH))
 
-        bitsLocalArtLoc = self.fetch_asset(self._bitsArtURL,
+        bitsLocalArtLoc = self.ASSET_BITS.fetch()
-                                           asset_hash=self._bitsArtSHA1Hash)
         self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLoc)
 
         # extract the bits artifact in the temp working directory
@@ -369,7 +372,7 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
         """The main test case implementation."""
 
         iso_file = os.path.join(self._workDir,
-                                'bits-%d.iso' %self._bitsInternalVer)
+                                'bits-%d.iso' %self.BITS_INTERNAL_VER)
 
         self.assertTrue(os.access(iso_file, os.R_OK))
 
@@ -393,12 +396,6 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
         self._vm.set_console()
         self._vm.launch()
 
-        self.logger.debug("Console output from bits VM follows ...")
-        c_drainer = drainer.LineLogger(self._vm.console_socket.fileno(),
-                                       logger=self.logger.getChild("console"),
-                                       stop_check=(lambda :
-                                                   not self._vm.is_running()))
-        c_drainer.start()
 
         # biosbits has been configured to run all the specified test suites
         # in batch mode and then automatically initiate a vm shutdown.
@@ -406,4 +403,8 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
         # with the avocado test timeout.
         self._vm.event_wait('SHUTDOWN', timeout=BITS_TIMEOUT)
         self._vm.wait(timeout=None)
+        self.logger.debug("Checking console output ...")
         self.parse_log()
+
+if __name__ == '__main__':
+    QemuBaseTest.main()