CONNECTION ERROT

[completeqwq]

raise RuntimeError("Cluster is not available after multiple attempts")
RuntimeError: Cluster is not available after multiple attempts

[nikelius]

我在Windows下也遇到过类似问题（异常栈见后）
个人判断该问题的原因：缺省防火墙等网络限制了一些端口的访问，导致管理服务无法正常连接worker进程。
（其实不算inference的锅哈）

当时的环境：Win10服务器
详细异常栈：
2024-07-04 14:34:53,905 xinference.core.worker 19788 INFO Purge cache directory: C:\Users\admin.xinference\cache
Traceback (most recent call last):
File "G:\miniconda\envs\py310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "G:\miniconda\envs\py310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "G:\miniconda\envs\py310\Scripts\xinference-local.exe_main.py", line 7, in
sys.exit(local())
File "G:\miniconda\envs\py310\lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "G:\miniconda\envs\py310\lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "G:\miniconda\envs\py310\lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "G:\miniconda\envs\py310\lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "G:\miniconda\envs\py310\lib\site-packages\xinference\deploy\cmdline.py", line 225, in local
start_local_cluster(
File "G:\miniconda\envs\py310\lib\site-packages\xinference\deploy\cmdline.py", line 112, in start_local_cluster
main(
File "G:\miniconda\envs\py310\lib\site-packages\xinference\deploy\local.py", line 122, in main
raise RuntimeError("Cluster is not available after multiple attempts")
RuntimeError: Cluster is not available after multiple attempts

[completeqwq]

我在Windows下也遇到过类似问题（异常栈见后） 个人判断该问题的原因：缺省防火墙等网络限制了一些端口的访问，导致管理服务无法正常连接worker进程。 （其实不算inference的锅哈）

当时的环境：Win10服务器 详细异常栈： 2024-07-04 14:34:53,905 xinference.core.worker 19788 INFO Purge cache directory: C:\Users\admin.xinference\cache Traceback (most recent call last): File "G:\miniconda\envs\py310\lib\runpy.py", line 196, in _run_module_as_main return run_code(code, main_globals, None, File "G:\miniconda\envs\py310\lib\runpy.py", line 86, in run_code exec(code, run_globals) File "G:\miniconda\envs\py310\Scripts\xinference-local.exe__main.py", line 7, in sys.exit(local()) File "G:\miniconda\envs\py310\lib\site-packages\click\core.py", line 1157, in call return self.main(*args, **kwargs) File "G:\miniconda\envs\py310\lib\site-packages\click\core.py", line 1078, in main rv = self.invoke(ctx) File "G:\miniconda\envs\py310\lib\site-packages\click\core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "G:\miniconda\envs\py310\lib\site-packages\click\core.py", line 783, in invoke return __callback(*args, **kwargs) File "G:\miniconda\envs\py310\lib\site-packages\xinference\deploy\cmdline.py", line 225, in local start_local_cluster( File "G:\miniconda\envs\py310\lib\site-packages\xinference\deploy\cmdline.py", line 112, in start_local_cluster main( File "G:\miniconda\envs\py310\lib\site-packages\xinference\deploy\local.py", line 122, in main raise RuntimeError("Cluster is not available after multiple attempts") RuntimeError: Cluster is not available after multiple attempts

请问怎么解决的

[nikelius]

请问怎么解决的

其实没解决，windows下稀奇古怪的问题太多，准备尝试wsl方式。
整体来说，服务启动涉及三个端口（supervisor、server，metrics）把这几个端口在防火墙上放开就好。
之前的思路是：1.关闭windows防火墙；2、单独放开某个端口的限制。
方法1，我的场景无法用（其他团队在用改主机，不让关闭防火墙）
方法2，如果是固定端口还好办，用下面命令可以放开某个指定端口：
netsh advfirewall firewall add rule name=WSL2api dir=in action=allow protocol=TCP localport=9997
xinference-local --host 0.0.0.0 --port 9997

但是xinference采用的动态端口机制，每次启动所使用的内部端口不一致，这就没法弄了。
第一次：64719 、65264
2024-07-10 14:56:23,283 xinference.core.supervisor 25764 INFO Xinference supervisor 0.0.0.0:64719 started
2024-07-10 14:56:23,621 xinference.core.worker 25764 INFO Starting metrics export server at 0.0.0.0:None
2024-07-10 14:56:23,624 xinference.core.worker 25764 INFO Checking metrics export server...
2024-07-10 14:56:31,034 xinference.core.worker 25764 INFO Metrics server is started at: http://0.0.0.0:65264
2024-07-10 14:56:31,037 xinference.core.worker 25764 INFO Xinference worker 0.0.0.0:64719 started
2024-07-10 14:56:31,038 xinference.core.worker 25764 INFO Purge cache directory: C:\Users\admin.xinference\cache
第二次：53127、65282
2024-07-10 14:59:09,471 xinference.core.supervisor 14648 INFO Xinference supervisor 0.0.0.0:53127 started
2024-07-10 14:59:09,722 xinference.core.worker 14648 INFO Starting metrics export server at 0.0.0.0:None
2024-07-10 14:59:09,737 xinference.core.worker 14648 INFO Checking metrics export server...
2024-07-10 14:59:16,456 xinference.core.worker 14648 INFO Metrics server is started at: http://0.0.0.0:65282
2024-07-10 14:59:16,459 xinference.core.worker 14648 INFO Xinference worker 0.0.0.0:53127 started
2024-07-10 14:59:16,461 xinference.core.worker 14648 INFO Purge cache directory: C:\Users\admin.xinference\cache

[nikelius]

补充：

本地模式（只固定了Metrics 端口，而supervisor 和 worker 是动态的，不解决问题）

命令：xinference-local --host 0.0.0.0 --port 9997 -MH 0.0.0.0 -mp 9996

2024-07-10 15:59:13,111 xinference.core.supervisor 26396 INFO Xinference supervisor 0.0.0.0:48462 started
2024-07-10 15:59:13,330 xinference.core.worker 26396 INFO Starting metrics export server at 0.0.0.0:9996
2024-07-10 15:59:13,345 xinference.core.worker 26396 INFO Checking metrics export server...
2024-07-10 15:59:20,456 xinference.core.worker 26396 INFO Metrics server is started at: http://0.0.0.0:9996
2024-07-10 15:59:20,460 xinference.core.worker 26396 INFO Xinference worker 0.0.0.0:48462 started
2024-07-10 15:59:20,461 xinference.core.worker 26396 INFO Purge cache directory: C:\Users\admin.xinference\cache

分布模式（supervisor 启动完成，但worker 启动失败）

命令：xinference-supervisor --host 10.10.83.104 --port 9997 --supervisor-port 9996

2024-07-10 16:00:11,564 xinference.core.supervisor 19516 INFO Xinference supervisor 10.10.83.104:9996 started
2024-07-10 16:00:18,706 xinference.api.restful_api 8172 INFO Starting Xinference at endpoint: http://10.10.83.104:9997

命令：xinference-worker -e "http://10.10.83.104:9997" --metrics-exporter-host 0.0.0.0 --metrics-exporter-port 9995

2024-07-10 16:01:24,061 xinference.core.worker 6844 INFO Starting metrics export server at 0.0.0.0:9995
2024-07-10 16:01:24,064 xinference.core.worker 6844 INFO Checking metrics export server...
2024-07-10 16:01:31,205 xinference.core.worker 6844 INFO Metrics server is started at: http://0.0.0.0:**9995**
OSError: [address=10.10.83.104:9996, pid=19516] [WinError 1214] 指定的网络名格式无效。

[nikelius]

请问怎么解决的

已解决
@completeqwq 修改启动命令，使用具体的IP地址替代0.0.0.0 就可以了。
原来：xinference-local --host 0.0.0.0 --port 9997
改为：xinference-local --host 10.10.83.104 --port 9997
放开防火墙端口限制：netsh advfirewall firewall add rule name=WSL2api dir=in action=allow protocol=TCP localport=9997

[github-actions]

This issue is stale because it has been open for 7 days with no activity.