Description
Your current environment
‘DevicePtrInfo getPointer(PyObject*, int)’:
/tmp/tmpp8zry39f/main.c:118:20: error: invalid conversion from ‘int’ to ‘CUresult’ {aka ‘cudaError_enum’} [-fpermissive]
118 | CUDA_CHECK(status); // Catch any other cuda API errors
| ^
| |
| int
/tmp/tmpp8zry39f/main.c:24:38: note: in definition of macro ‘CUDA_CHECK’
24 | #define CUDA_CHECK(ans) { gpuAssert((ans), FILE, LINE); }
| ^~~
/tmp/tmpp8zry39f/main.c:7:39: note: initializing argument 1 of ‘void gpuAssert(CUresult, const char*, int)’
7 | static inline void gpuAssert(CUresult code, const char *file, int line)
| ~~~~~~~~~^~~~
ERROR 05-10 09:24:36 [core.py:387] EngineCore hit an exception: Traceback (most recent call last):
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 378, in run_engine_core
ERROR 05-10 09:24:36 [core.py:387] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 320, in init
ERROR 05-10 09:24:36 [core.py:387] super().init(vllm_config, executor_class, log_stats)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 71, in init
ERROR 05-10 09:24:36 [core.py:387] self._initialize_kv_caches(vllm_config)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 133, in _initialize_kv_caches
ERROR 05-10 09:24:36 [core.py:387] available_gpu_memory = self.model_executor.determine_available_memory()
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/v1/executor/abstract.py", line 66, in determine_available_memory
ERROR 05-10 09:24:36 [core.py:387] output = self.collective_rpc("determine_available_memory")
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/executor/uniproc_executor.py", line 56, in collective_rpc
ERROR 05-10 09:24:36 [core.py:387] answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/utils.py", line 2378, in run_method
ERROR 05-10 09:24:36 [core.py:387] return func(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 05-10 09:24:36 [core.py:387] return func(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 157, in determine_available_memory
ERROR 05-10 09:24:36 [core.py:387] self.model_runner.profile_run()
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 1591, in profile_run
ERROR 05-10 09:24:36 [core.py:387] hidden_states = self._dummy_run(self.max_num_tokens)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 05-10 09:24:36 [core.py:387] return func(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 1441, in _dummy_run
ERROR 05-10 09:24:36 [core.py:387] hidden_states = model(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
ERROR 05-10 09:24:36 [core.py:387] return self._call_impl(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
ERROR 05-10 09:24:36 [core.py:387] return forward_call(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/model_executor/models/qwen3.py", line 301, in forward
ERROR 05-10 09:24:36 [core.py:387] hidden_states = self.model(input_ids, positions, intermediate_tensors,
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/compilation/decorators.py", line 238, in call
ERROR 05-10 09:24:36 [core.py:387] output = self.compiled_callable(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 574, in _fn
ERROR 05-10 09:24:36 [core.py:387] return fn(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1380, in call
ERROR 05-10 09:24:36 [core.py:387] return self._torchdynamo_orig_callable(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 547, in call
ERROR 05-10 09:24:36 [core.py:387] return _compile(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 986, in _compile
ERROR 05-10 09:24:36 [core.py:387] guarded_code = compile_inner(code, one_graph, hooks, transform)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 715, in compile_inner
ERROR 05-10 09:24:36 [core.py:387] return _compile_inner(code, one_graph, hooks, transform)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_utils_internal.py", line 95, in wrapper_function
ERROR 05-10 09:24:36 [core.py:387] return function(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 750, in _compile_inner
ERROR 05-10 09:24:36 [core.py:387] out_code = transform_code_object(code, transform)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1361, in transform_code_object
ERROR 05-10 09:24:36 [core.py:387] transformations(instructions, code_options)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 231, in _fn
ERROR 05-10 09:24:36 [core.py:387] return fn(*args, **kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 662, in transform
ERROR 05-10 09:24:36 [core.py:387] tracer.run()
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2868, in run
ERROR 05-10 09:24:36 [core.py:387] super().run()
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1052, in run
ERROR 05-10 09:24:36 [core.py:387] while self.step():
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 962, in step
ERROR 05-10 09:24:36 [core.py:387] self.dispatch_table[inst.opcode](self, inst)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3048, in RETURN_VALUE
ERROR 05-10 09:24:36 [core.py:387] self._return(inst)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3033, in _return
ERROR 05-10 09:24:36 [core.py:387] self.output.compile_subgraph(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1101, in compile_subgraph
ERROR 05-10 09:24:36 [core.py:387] self.compile_and_call_fx_graph(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1382, in compile_and_call_fx_graph
ERROR 05-10 09:24:36 [core.py:387] compiled_fn = self.call_user_compiler(gm)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1432, in call_user_compiler
ERROR 05-10 09:24:36 [core.py:387] return self._call_user_compiler(gm)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1483, in _call_user_compiler
ERROR 05-10 09:24:36 [core.py:387] raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1462, in _call_user_compiler
ERROR 05-10 09:24:36 [core.py:387] compiled_fn = compiler_fn(gm, self.example_inputs())
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/dynamo/repro/after_dynamo.py", line 130, in call
ERROR 05-10 09:24:36 [core.py:387] compiled_gm = compiler_fn(gm, example_inputs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/dynamo/repro/after_dynamo.py", line 130, in call
ERROR 05-10 09:24:36 [core.py:387] compiled_gm = compiler_fn(gm, example_inputs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/init.py", line 2385, in call
ERROR 05-10 09:24:36 [core.py:387] return self.compiler_fn(model, inputs, **self.kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/compilation/backends.py", line 455, in call
ERROR 05-10 09:24:36 [core.py:387] PiecewiseCompileInterpreter(self.split_gm, submod_names_to_compile,
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/compilation/backends.py", line 245, in run
ERROR 05-10 09:24:36 [core.py:387] return super().run(*fake_args)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/fx/interpreter.py", line 167, in run
ERROR 05-10 09:24:36 [core.py:387] self.env[node] = self.run_node(node)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/fx/interpreter.py", line 230, in run_node
ERROR 05-10 09:24:36 [core.py:387] return getattr(self, n.op)(n.target, args, kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/compilation/backends.py", line 260, in call_module
ERROR 05-10 09:24:36 [core.py:387] compiled_graph_for_general_shape = self.vllm_backend.
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/compilation/backends.py", line 121, in compile
ERROR 05-10 09:24:36 [core.py:387] compiled_graph, handle = self.compiler.compile(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/compilation/compiler_interface.py", line 293, in compile
ERROR 05-10 09:24:36 [core.py:387] compiled_graph = compile_fx(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1552, in compile_fx
ERROR 05-10 09:24:36 [core.py:387] return compile_fx(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1863, in compile_fx
ERROR 05-10 09:24:36 [core.py:387] return aot_autograd(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/backends/common.py", line 83, in call
ERROR 05-10 09:24:36 [core.py:387] cg = aot_module_simplified(gm, example_inputs, **self.kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1155, in aot_module_simplified
ERROR 05-10 09:24:36 [core.py:387] compiled_fn = dispatch_and_compile()
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1131, in dispatch_and_compile
ERROR 05-10 09:24:36 [core.py:387] compiled_fn, _ = create_aot_dispatcher_function(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 580, in create_aot_dispatcher_function
ERROR 05-10 09:24:36 [core.py:387] return _create_aot_dispatcher_function(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 830, in _create_aot_dispatcher_function
ERROR 05-10 09:24:36 [core.py:387] compiled_fn, fw_metadata = compiler_fn(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py", line 203, in aot_dispatch_base
ERROR 05-10 09:24:36 [core.py:387] compiled_fw = compiler(fw_module, updated_flat_args)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 489, in call
ERROR 05-10 09:24:36 [core.py:387] return self.compiler_fn(gm, example_inputs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1741, in fw_compiler_base
ERROR 05-10 09:24:36 [core.py:387] return inner_compile(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/contextlib.py", line 79, in inner
ERROR 05-10 09:24:36 [core.py:387] return func(*args, **kwds)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/vllm/compilation/compiler_interface.py", line 228, in hijacked_compile_fx_inner
ERROR 05-10 09:24:36 [core.py:387] output = torch._inductor.compile_fx.compile_fx_inner(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 569, in compile_fx_inner
ERROR 05-10 09:24:36 [core.py:387] return wrap_compiler_debug(_compile_fx_inner, compiler_name="inductor")(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_dynamo/repro/after_aot.py", line 102, in debug_wrapper
ERROR 05-10 09:24:36 [core.py:387] inner_compiled_fn = compiler_fn(gm, example_inputs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 685, in _compile_fx_inner
ERROR 05-10 09:24:36 [core.py:387] mb_compiled_graph = fx_codegen_and_compile(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1129, in fx_codegen_and_compile
ERROR 05-10 09:24:36 [core.py:387] return scheme.codegen_and_compile(gm, example_inputs, inputs_to_check, graph_kwargs)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1044, in codegen_and_compile
ERROR 05-10 09:24:36 [core.py:387] compiled_fn = graph.compile_to_module().call
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/graph.py", line 2027, in compile_to_module
ERROR 05-10 09:24:36 [core.py:387] return self._compile_to_module()
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/graph.py", line 2068, in _compile_to_module
ERROR 05-10 09:24:36 [core.py:387] mod = PyCodeCache.load_by_key_path(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/codecache.py", line 2759, in load_by_key_path
ERROR 05-10 09:24:36 [core.py:387] mod = _reload_python_module(key, path)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/runtime/compile_tasks.py", line 45, in _reload_python_module
ERROR 05-10 09:24:36 [core.py:387] exec(code, mod.dict, mod.dict)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/.cache/vllm/torch_compile_cache/fc89714a6f/rank_0_0/inductor_cache/si/csihqjvydc332maaf7scpbpdymkxexmsthfwpc73z5wc4swzehvz.py", line 66, in
ERROR 05-10 09:24:36 [core.py:387] triton_red_fused__to_copy_add_embedding_mean_mul_pow_rsqrt_0 = async_compile.triton('triton_red_fused__to_copy_add_embedding_mean_mul_pow_rsqrt_0', '''
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/async_compile.py", line 213, in triton
ERROR 05-10 09:24:36 [core.py:387] kernel.precompile()
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/runtime/triton_heuristics.py", line 293, in precompile
ERROR 05-10 09:24:36 [core.py:387] compiled_binary, launcher = self._precompile_config(
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/torch/_inductor/runtime/triton_heuristics.py", line 520, in _precompile_config
ERROR 05-10 09:24:36 [core.py:387] binary._init_handles()
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/triton/compiler/compiler.py", line 384, in _init_handles
ERROR 05-10 09:24:36 [core.py:387] self.run = driver.active.launcher_cls(self.src, self.metadata)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/triton/backends/nvidia/driver.py", line 440, in init
ERROR 05-10 09:24:36 [core.py:387] mod = compile_module_from_src(src, "__triton_launcher")
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/triton/backends/nvidia/driver.py", line 57, in compile_module_from_src
ERROR 05-10 09:24:36 [core.py:387] so = _build(name, src_path, tmpdir, library_dirs(), include_dir, libraries)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/triton/runtime/build.py", line 50, in _build
ERROR 05-10 09:24:36 [core.py:387] ret = subprocess.check_call(cc_cmd)
ERROR 05-10 09:24:36 [core.py:387] File "/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/subprocess.py", line 369, in check_call
ERROR 05-10 09:24:36 [core.py:387] raise CalledProcessError(retcode, cmd)
ERROR 05-10 09:24:36 [core.py:387] torch._dynamo.exc.BackendCompilerFailed: backend='<vllm.compilation.backends.VllmBackend object at 0x770ad07f48b0>' raised:
ERROR 05-10 09:24:36 [core.py:387] CalledProcessError: Command '['/usr/bin/gcc', '/tmp/tmpp8zry39f/main.c', '-O3', '-shared', '-fPIC', '-Wno-psabi', '-o', '/tmp/tmpp8zry39f/__triton_launcher.cpython-310-x86_64-linux-gnu.so', '-lcuda', '-L/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/triton/backends/nvidia/lib', '-L/lib/x86_64-linux-gnu', '-L/lib/i386-linux-gnu', '-I/home/house365ai/anaconda3/envs/vllm6/lib/python3.10/site-packages/triton/backends/nvidia/include', '-I/tmp/tmpp8zry39f', '-I/home/house365ai/anaconda3/envs/vllm6/include/python3.10']' returned non-zero exit status 1.
ERROR 05-10 09:24:36 [core.py:387]
ERROR 05-10 09:24:36 [core.py:387] Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
ERROR 05-10 09:24:36 [core.py:387]
ERROR 05-10 09:24:36 [core.py:387]
ERROR 05-10 09:24:36 [core.py:387] You can suppress this exception and fall back to eager by setting:
ERROR 05-10 09:24:36 [core.py:387] import torch._dynamo
ERROR 05-10 09:24:36 [core.py:387] torch._dynamo.config.suppress_errors = True
ERROR 05-10 09:24:36 [core.py:387]
ERROR 05-10 09:24:36 [core.py:387]
CRITICAL 05-10 09:24:36 [core_client.py:359] Got fatal signal from worker processes, shutting down. See stack trace above for root cause issue.
Killed
How you are installing vllm
pip install -vvv vllmBefore submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.