history: max flush count uint16 overflow #308
Description
A electrumx instance I run on the mutinynet has a History:flush count: 65,535 and is now rejecting to start as the max size of the uint16 is reached to which the variable is converted:
electrumx/src/electrumx/server/history.py
Line 160 in be58fd8
The simple solution seems to be to increase the size of flush count to a uint32 but the flush count is part of the db keys which would be affected by this.
Certificate files already exist. Not generating new ones.
INFO:electrumx:ElectrumX server starting. (spesmilo. version=1.18.0)
INFO:electrumx:logging level: INFO
INFO:Controller:Python version: 3.11.2 (main, Apr 28 2025, 14:11:48) [GCC 12.2.0]
INFO:Controller:software version: ElectrumX 1.18.0
INFO:Controller:aiorpcX version: 0.25.0
INFO:Controller:supported protocol versions: 1.4-1.4.3
INFO:Controller:event loop policy: None
INFO:Controller:reorg limit is 8,000 blocks
INFO:Daemon:daemon #1 at bitcoind:38332/ (current)
INFO:DB:switching current directory to /data
INFO:DB:using leveldb for DB backend
INFO:DB:opened UTXO DB (for sync: True)
INFO:DB:UTXO DB version: 8
INFO:DB:coin: Bitcoin
INFO:DB:network: mutinynet
INFO:DB:height: 2,426,940
INFO:DB:tip: 000000508ac6b9cfb1b23ea20ae84f74c82fcfbb51bb8b337dc5e62a9bba8357
INFO:DB:tx count: 7,113,645
INFO:DB:flushing DB cache at 1,200 MB
INFO:History:history DB version: 1
INFO:History:flush count: 65,535
INFO:Prefetcher:catching up to daemon height 2,430,331 (3,391 blocks behind)
INFO:SessionManager:RPC server listening on localhost:8000
INFO:BlockProcessor:our height: 2,426,950 daemon: 2,430,331 UTXOs 0MB hist 0MB
INFO:BlockProcessor:processed 10 blocks size 0.06 MB in 0.0s
INFO:BlockProcessor:processed 100 blocks size 0.57 MB in 0.0s
INFO:BlockProcessor:processed 200 blocks size 0.76 MB in 0.0s
INFO:BlockProcessor:processed 300 blocks size 1.45 MB in 0.1s
INFO:BlockProcessor:processed 700 blocks size 1.56 MB in 0.1s
INFO:BlockProcessor:processed 600 blocks size 2.85 MB in 0.1s
INFO:BlockProcessor:processed 1,000 blocks size 3.54 MB in 0.1s
INFO:BlockProcessor:processed 481 blocks size 2.16 MB in 0.1s
INFO:BlockProcessor:caught up to height 2430331
INFO:DB:flushed filesystem data in 0.00s
INFO:Prefetcher:cancelled; prefetcher stopping
INFO:SessionManager:closing down server for rpc://localhost:8000
INFO:Controller:shutting down
INFO:Controller:shutdown complete
ERROR:electrumx:ElectrumX server terminated abnormally
Traceback (most recent call last):
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/cli/electrumx_server.py", line 38, in main
asyncio.run(controller.run())
File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/lib/server_base.py", line 129, in run
await server_task
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/lib/server_base.py", line 102, in serve
await self.serve(shutdown_event)
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/controller.py", line 141, in serve
async with OldTaskGroup() as group:
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/aiorpcx/curio.py", line 304, in __aexit__
await self.join()
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/lib/util.py", line 371, in join
task.result()
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/block_processor.py", line 712, in fetch_and_process_blocks
async with OldTaskGroup() as group:
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/aiorpcx/curio.py", line 304, in __aexit__
await self.join()
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/lib/util.py", line 371, in join
task.result()
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/block_processor.py", line 666, in _process_prefetched_blocks
await self._first_caught_up()
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/block_processor.py", line 682, in _first_caught_up
await self.flush(True)
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/block_processor.py", line 368, in flush
await self.run_in_thread_with_lock(flush)
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/block_processor.py", line 222, in run_in_thread_with_lock
return await asyncio.shield(run_in_thread_locked())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/block_processor.py", line 221, in run_in_thread_locked
return await run_in_thread(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/aiorpcx/curio.py", line 57, in run_in_thread
return await get_event_loop().run_in_executor(None, func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/block_processor.py", line 366, in flush
self.db.flush_dbs(self.flush_data(), flush_utxos,
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/db.py", line 243, in flush_dbs
self.flush_history()
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/db.py", line 312, in flush_history
self.history.flush()
File "/root/.local/pipx/venvs/e-x/lib/python3.11/site-packages/electrumx/server/history.py", line 160, in flush
flush_id = pack_be_uint16(self.flush_count)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: 'H' format requires 0 <= number <= 65535