Skip to content

Commit c65e79f

Browse files
committed
use new unique jobs implementation
This moves the library to use the new unique jobs implementation from riverqueue/river#590 and migrates the sqlalchemy driver to use a unified insertion path, allowing bulk inserts to use unique jobs.
1 parent fc1a2b8 commit c65e79f

19 files changed

+528
-1603
lines changed

README.md

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,8 @@ class JobArgs(Protocol):
4848
pass
4949
```
5050

51-
* `kind` is a unique string that identifies them the job in the database, and which a Go worker will recognize.
52-
* `to_json()` defines how the job will serialize to JSON, which of course will have to be parseable as an object in Go.
51+
- `kind` is a unique string that identifies them the job in the database, and which a Go worker will recognize.
52+
- `to_json()` defines how the job will serialize to JSON, which of course will have to be parseable as an object in Go.
5353

5454
They may also respond to `insert_opts()` with an instance of `InsertOpts` to define insertion options that'll be used for all jobs of the kind.
5555

@@ -95,16 +95,6 @@ insert_res.job
9595
insert_res.unique_skipped_as_duplicated
9696
```
9797

98-
### Custom advisory lock prefix
99-
100-
Unique job insertion takes a Postgres advisory lock to make sure that its uniqueness check still works even if two conflicting insert operations are occurring in parallel. Postgres advisory locks share a global 64-bit namespace, which is a large enough space that it's unlikely for two advisory locks to ever conflict, but to _guarantee_ that River's advisory locks never interfere with an application's, River can be configured with a 32-bit advisory lock prefix which it will use for all its locks:
101-
102-
```python
103-
client = riverqueue.Client(riversqlalchemy.Driver(engine), advisory_lock_prefix=123456)
104-
```
105-
106-
Doing so has the downside of leaving only 32 bits for River's locks (64 bits total - 32-bit prefix), making them somewhat more likely to conflict with each other.
107-
10898
## Inserting jobs in bulk
10999

110100
Use `#insert_many` to bulk insert jobs as a single operation for improved efficiency:

requirements-dev.lock

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
# all-features: false
88
# with-sources: false
99
# generate-hashes: false
10+
# universal: false
1011

1112
-e file:.
1213
asyncpg==0.29.0

requirements.lock

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
# all-features: false
88
# with-sources: false
99
# generate-hashes: false
10+
# universal: false
1011

1112
-e file:.
1213
sqlalchemy==2.0.30

0 commit comments

Comments
 (0)