YOLO训练报错

发布于:2024-12-07 ⋅ 阅读:(98) ⋅ 点赞:(0)

Traceback (most recent call last):
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/ultralytics/engine/model.py", line 802, in train
    self.trainer.train()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/ultralytics/engine/trainer.py", line 207, in train
    self._do_train(world_size)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/ultralytics/engine/trainer.py", line 361, in _do_train
    self.train_loader.reset()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/ultralytics/data/build.py", line 56, in reset
    self.iterator = self._get_iterator()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 388, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1038, in __init__
    w.start()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/popen_fork.py", line 68, in _launch
    parent_r, child_w = os.pipe()
OSError: [Errno 24] Too many open files
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/threading.py", line 870, in run
    self.run()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/torch/utils/data/_utils/pin_memory.py", line 55, in _pin_memory_loop
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/torch/utils/data/_utils/pin_memory.py", line 55, in _pin_memory_loop
    do_one_step()
    do_one_step()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/torch/utils/data/_utils/pin_memory.py", line 32, in do_one_step
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/torch/utils/data/_utils/pin_memory.py", line 32, in do_one_step
    r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/queues.py", line 116, in get
    r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/queues.py", line 116, in get
    return _ForkingPickler.loads(res)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/torch/multiprocessing/reductions.py", line 496, in rebuild_storage_fd
    return _ForkingPickler.loads(res)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/site-packages/torch/multiprocessing/reductions.py", line 496, in rebuild_storage_fd
    fd = df.detach()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/resource_sharer.py", line 57, in detach
    fd = df.detach()
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/resource_sharer.py", line 87, in get_connection
    with _resource_sharer.get_connection(self._id) as conn:
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/connection.py", line 502, in Client
    c = Client(address, authkey=process.current_process().authkey)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/connection.py", line 508, in Client
    answer_challenge(c, authkey)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/connection.py", line 752, in answer_challenge
    c = SocketClient(address)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
    message = connection.recv_bytes(256)         # reject large message
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/connection.py", line 216, in recv_bytes
    s.connect(address)
    buf = self._recv_bytes(maxlength)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
ConnectionRefusedError: [Errno 111] Connection refused
    buf = self._recv(4)
  File "/home/dell/anaconda3/envs/ultralytics/lib/python3.8/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

解决办法

增加系统打开文件限制

默认情况下,操作系统对每个进程可以打开的文件数有一个限制。你可以通过增加这个限制来防止错误。

使用以下命令检查当前文件描述符限制:

ulimit -n

然后可以增加文件描述符的限制:

ulimit -n 4096 # 将限制增加到 4096 或更高

如果你希望每次开机都生效,可以将此命令添加到你的 shell 配置文件(如 .bashrc.zshrc)中:

ulimit -n 4096


网站公告

今日签到

点亮在社区的每一天
去签到