vector-store: Move Usearch operations to a dedicated tasks #294
Conversation
knowack1
force-pushed
the
vector-339-vector-store-blocked-during-fullscan
branch
2 times, most recently
from
7878e6a to
4d97b34
Compare
knowack1
changed the title
| .take(limit.0.get()) | ||
| .collect(); | ||
|
|
||
| sim.wait_search(start); |
There was a problem hiding this comment.
We should measure time of preparing response also, so wait_search should be after preparing keys vector.
There was a problem hiding this comment.
Change reverted.
I was changing simulator because I tokio was handing for some reason - I though it was due to locks.
But in fact to was due to dropped tx channel for config in tests: https://github.com/scylladb/vector-store/pull/294/files#diff-cb2126eb464b326d11b2dcfbd02661e90d1a34a8735bd45a911c0e2bfee2607dL79. This caused that rx side was in the "busy loop".
There was a problem hiding this comment.
It is not reverted. The commit is still there.
| let keys = sim.keys.read().unwrap(); | ||
| let len = keys.len() as u64; |
There was a problem hiding this comment.
we don't need to lock sim.keys for all time of finding random keys.
There was a problem hiding this comment.
Fixed.
Now keys are locked temporary only.
knowack1
force-pushed
the
vector-339-vector-store-blocked-during-fullscan
branch
from
4d97b34 to
50ab25b
Compare
The simulator crashes if a search was performed on an empty index (i.e. before any keys were indexed).
knowack1
force-pushed
the
vector-339-vector-store-blocked-during-fullscan
branch
from
50ab25b to
e056901
Compare
Usearch's API uses global locks. Operations like `add` and `remove` can hold these locks for extended periods. When these operations are executed on the Tokio runtime, they can block worker threads. This leads to thread starvation, effectively halting all asynchronous tasks, including handling HTTP requests, processing CDC data, and serving metrics. This commit moves the blocking `add` and `remove` calls to a dedicated Rayon thread pool. By isolating these operations, the Tokio runtime remains unblocked and responsive, ensuring the application's I/O continues to function correctly.
knowack1
force-pushed
the
vector-339-vector-store-blocked-during-fullscan
branch
from
e056901 to
f56877a
Compare
ewienik
left a comment
ewienik
left a comment
There was a problem hiding this comment.
There is a need for limiting rayon background threads to the same number as tokio threads.
| } => { | ||
| add_or_replace(idx, keys, usearch_key, primary_key, embedding); | ||
| rayon::spawn(move || { | ||
| add_or_replace(idx, keys, usearch_key, primary_key, embedding); |
There was a problem hiding this comment.
currently idx is based on RwLock, so there will be still blocking of thread. Better would be to replace the RwLock with a semaphore.
2 participants
Usearch's API uses global locks. Operations like
addandremovecan hold these locks for extended periods.When these operations are executed on the Tokio runtime, they can block
worker threads. This leads to thread starvation, effectively halting all
asynchronous tasks, including handling HTTP requests, processing CDC
data, and serving metrics.
This commit moves the blocking
addandremovecalls to a dedicatedRayon thread pool. By isolating these operations, the Tokio runtime
remains unblocked and responsive, ensuring
the application's I/O continues to function correctly.
Fixes: VECTOR-339