Skip to content

Conversation

@haardikk21
Copy link
Contributor

@haardikk21 haardikk21 commented Sep 2, 2025

Adds periodic dumping of DB data into S3, serialized as Parquet records

  • Saves up to retention_blocks worth of block data in postgres. Anything older than that gets dumped into S3
  • Jobs are block aligned based on range, so can be run across multiple instances
  • Need to acquire lock on processing a specific job, so multiple workers will not do overlapping repeated work

@cb-heimdall
Copy link
Collaborator

cb-heimdall commented Sep 2, 2025

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 0
Sum 1

@haardikk21 haardikk21 marked this pull request as draft September 2, 2025 17:06
@haardikk21 haardikk21 marked this pull request as ready for review September 2, 2025 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants