[BUG] 3007.1 metadata grains fails to load with minion

[r-mettner-skaylink]

Description

metadata_server_grains is not working under 3007.1

Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)

Salt 3007.1 minion.

• AWS EC2 Instances running ubuntu 22.04
• onedir packaging, installed from official repository

Steps to Reproduce the behavior
set metadata_server_grains: true in minion config.
Run minion (exception can be seen when running salt-minion with -ldebug)
Issue a grains.get meta-data from salt-master

Expected behavior
grains.get meta-data returning metadata grains.

[ERROR   ] Exception raised when processing __virtual__ function for salt.loaded.int.grains.metadata. Module will not be loaded: Cannot run the event loop while another loop is running
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1174, in _process_virtual
    virtual = self.run(virtual_attr)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1269, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1284, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/grains/metadata.py", line 38, in __virtual__
    if http.query(os.path.join(HOST, "latest/"), status=True).get("status") != 200:
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/http.py", line 633, in query
    HTTPClient(max_body_size=max_body)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/tornado/httpclient.py", line 109, in __init__
    self._async_client = self._io_loop.run_sync(make_client)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/tornado/ioloop.py", line 521, in run_sync
    self.start()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 195, in start
    self.asyncio_loop.run_forever()
  File "/opt/saltstack/salt/lib/python3.10/asyncio/base_events.py", line 592, in run_forever
    self._check_running()
  File "/opt/saltstack/salt/lib/python3.10/asyncio/base_events.py", line 586, in _check_running
    raise RuntimeError(
RuntimeError: Cannot run the event loop while another loop is running

Versions Report

#minion
Salt Version:
          Salt: 3007.1
 
Python Version:
        Python: 3.10.14 (main, Apr  3 2024, 21:30:09) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.16.0
      cherrypy: 18.8.0
      dateutil: 2.9.0.post0
     docker-py: 7.1.0
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.4
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.16.0
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4
 
Salt Package Information:
  Package Type: onedir
 
System Versions:
          dist: ubuntu 22.04.5 jammy
        locale: utf-8
       machine: x86_64
       release: 6.8.0-1024-aws
        system: Linux
       version: Ubuntu 22.04.5 jammy
#master
Salt Version:
          Salt: 3007.1
 
Python Version:
        Python: 3.10.14 (main, Apr  3 2024, 21:33:04) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
      dateutil: 2.9.0.post0
     docker-py: 7.1.0
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.4
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.16.0
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4
 
Salt Package Information:
  Package Type: onedir
 
System Versions:
          dist: ubuntu 22.04.5 jammy
        locale: utf-8
       machine: aarch64
       release: 6.8.0-1018-aws
        system: Linux
       version: Ubuntu 22.04.5 jammy