SKU/molecule #1234
Draft
SKU/molecule #1234
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…es w/o content_length (#1235)
…arlier than expected (#1237)
* Kicked off webhooks domain: described webhook subscription aggregate * Added 'WebhookEvent' simple aggregate * Sketched subscription event store methods * Sketched WebhookDelivery (fact) and it's repository * Couple more methods in delivery repository * Improved ID types. Introduced TaskAttemptId type * Introduced concept of `TaskAttempt` in task, and reshuffled state structure around it without changing event store * Drafter retrying logic and retry policies at task aggregate level. Not integrated or materialized yet. * Drafted retries in task snapshots * Working sqlite migration * Stabilized existing tests * Unit tests added for retry policies * Task scheduler config for retry policy + service-layer tests for retries * Implemented repository for webhook events (3 incarnations, no tests yet) * Unit tests implemented for WebhookEvent repositories. Detecting duplicate ID when creating events. * SQL for webhook subscriptions and deliveries * Drafted in-memory repo for webhook subscriptions * Sketched in-memory webhook delivery repository * Drafted Postgres/SQlite implementations fo webhook delivery repository * Drafted Postgres webhook subscription event store implementation * Implemented SQlite version of webhook subscription store * Tests for webhook delivery repository * Basic coverage for webhook subscription event store * Merge corrections * Finished tests for repository layer * Test corrections * Drafted webhook event bridge service * Sketched webhooks service layer, including: - webhook logical plan in Task System, propagation through planner/runner - webhook sender: populates delivery object, generates headers, sends webhook, updates the delivery object with webhook response - webhook signer: implementation of RFC9421 * udeps cleaned * Wrote very naive webhook signing and sender tests * Removed "ref" field from webhook event + linter fixed * WebhookSender => WebhookDeliveryWorker * Separated webhook outbox bride and event builder * Sending more universal DATASET.REF.UPDATED event with "blockRef" field * Separated WebhookSender from WebhookDelivery worker (to increase testability) * WebhookOutboxBridge => WebhookDeliveryScheduler * Drafted webhook secret generator * Added outbox consumer that removes subscriptions of a removed dataset * MInor correction in sending test * Sketched webhook subscriptions GQL api (no tests yet, incomplete) * Self-review question * Merge corrections * merge corrections * Removed task retries. Binding webhook delivery 1-to-1 to task * First test for subscriptions API * Extended test coverage for webhook subscription create GQL entry. Extracted creation use case * Reorganized GQL api handlers to use use cases more * Reorganized webhook subscriptions API to be more dataset bound to avoid separate security checking * Reshuffled update operations in GQL for webhook subscriptions * Idempotent subscription reactions * More subscription GQL API tests * Webhook subscription use case tests * Tests for WebhookDatasetRemovalHandler * Tests for webhook delivery scheduler service * Tests for webhook delivery worker * Merge corrections * Will lint/codegen pass with project toolchain? * try fixing lint/codegen * Removed obsolete comment * Recovered original formatting of GQL schema
* Custom workflow for SQLX migrations dev branch * GQL: Account Deletion API (#1242) * GQL: AccountMut::delete_account(): scaffolding * validate_password(): use .len() * DidSecretKeyRowModel: use type fullpath for DidEntityType * schema.gql: update * AccountMut::delete_account_by_name(): use DeleteAccountUseCase * AccountMut: use AccountName scalar instead of String * AccountMut: use Email scalar instead of String * AccountService::delete_account_by_name(): add * make sqlx-local-setup: fix * did_secret_keys: drop creator_id * CreateAccountUseCaseImpl::execute(): update fallback email generation * AccountService::delete_account_by_name(): add [2] * AccountRepository::delete_account_by_name(): implement * account_messages.rs -> account_lifecycle_message.rs * DeleteAccountUseCaseImpl: send AccountLifecycleMessage::deleted() * AccountRepository::delete_account_by_name(): return the removed account not only id * AccountLifecycleMessageDeleted::display_name: add * DatasetAccountDeletionHandler: introduce * DatasetRegistry::all_dataset_handles_by_owner_id(): implement * DatasetRegistry::all_dataset_handles_by_owner() -> all_dataset_handles_by_owner_name() * DatasetAccountDeletionHandler: introduce [2] * DidSecretKeyRepository::delete_did_secret_key(): add * kamu-account-services: group message producers and consumers * messaging_outbox::prelude: introduce * messaging_outbox::prelude: introduce * DidSecretService: handle DatasetLifecycleMessage * access_tokens: add ON DELETE CASCADE for account_id * accounts_passwords: add account_id * Linter fixes * CreateAccountUseCaseImpl::generate_email(): extract * PasswordHashRepository::save_password_hash(): add "account_id" * Makefile: resort vertically db crates * AccountLifecycleMessageDeleted: add "email" field * DidSecretKeyRepository::get_did_secret_key(): add * test_insert_and_locate_did_secret_keys(): update * test_create_account(): fixed * test_create_dataset_from_snapshot_creates_did_secret_key(): fixed * test_update_email_bad_email(): fixed * Makefile: remove db crate duplicates * sqlx: add cached queries * AccountsMut::create_account(): absorb * AccountMut::delete(): rethink * AccountMut: access checks * AccountMut: update tracings * DeleteAccountUseCaseImpl: only admins * GQL: AccountMut::modify_password(): re-think * DeleteAccountUseCaseImpl: allow self-deletion * Self-review * CHANGELOG.md: update * DeleteAccountUseCaseImpl::authenticated(): renamed from unauthenticated() * Integration fixes * SelfDeletionIsForbidden: remove unused struct * AccountRepository::delete_account_by_name(): do not return deleted account * OsoDatasetAuthorizer: move dill macros to struct declaration * DeleteAccountUseCase::execute(): take &Account as argument * DeleteAccountUseCaseImpl: use utils::AccountAuthorizationHelper * DatasetRegistry::all_dataset_handles_by_owner_id(): return odfOwnedDatasetHandleStream * Revert "DatasetRegistry::all_dataset_handles_by_owner_id(): return odfOwnedDatasetHandleStream" This reverts commit 8ac0da8. * DatasetAccountDeletionHandler::handle_account_lifecycle_deleted_message(): add a PERF note * CI: fix codegen action * CHANGELOG.md: update * test_delete_account_use_case_impl(): implement * GQL: test_accounts: add tests * CHANGELOG.md: update * Tests fixes * sqlx: update cached queries * Remove password logic from account service level (#1243) * Refactor password logic * Update changelog * GQL: Collection API, `extra_data` validation (#1246) * CollectionMut: take dataset by a ref * VersionedFileMut: take dataset by a ref * GQL: Add ExtraData scalar * GQL: use ExtraData scalar * Typo fixes * Fix tests * GQL: ExtraData scalar: add tests * CHANGELOG.md: update * Allow MIT-0 license usage * kamu-adapter-graphql: correct feature gate * Release (minor): 0.238.0 (#1248) --------- Co-authored-by: Sergei Zaychenko <[email protected]> Co-authored-by: Roman Boiko <[email protected]>
* Upgrade to new rustc and 2024 edition (#1254) * Search by account name (#1253) * Search filters also by account name * Update changelog * Fix review comments. Iter 1 * Replace format by to_string() * Fix fmt * Wallet based authentication: Phase 1 (#1239) * EvmWalletAuthenticationProvider: scaffolding * kamu-datasets: remove extra dep (itertools) * kamu-adapter-auth-web3: implement Web3WalletAuthenticationProvider (w/o nonce checking) * kamu --show-error-stack-trace * APIServerRunCommand: get token after HTTP server initialization * kamu-cli: activate Web3WalletAuthenticationProvider * kamu-adapter-graphql: extract auth_mut/ * kamu-adapter-auth-web3: ChecksumWalletAddress -> ChecksumEvmWalletAddress * GQL: AuthWeb3Mut::nonce(): implement * kamu-web3: introduce Web3AuthNonceRepository * kamu-web3-services: introduce Web3NonceServiceImpl * kamu-cli: register kamu-web3-services * kamu-auth-web3: update EIP_4361_EXPECTED_STATEMENT text * schema.gql: update * Web3NonceServiceImpl: impl InitOnStartup * kamu-web -> kamu-auth-web * kamu-auth-web3-inmem: implement * kamu-auth-web3-repo-tests: implement * kamu-auth-web3-inmem: tests * AuthWeb3Mut: nonce() -> eip4361_auth_nonce() * kamu-auth-web3-postgres: scaffolding * kamu-auth-web3-postgres: implement * Web3AuthenticationNonceEntity: expired_at -> expires_at * kamu-auth-web3-sqlite: implement * sqlx: update cached queries * Web3AuthenticationNonce -> Web3AuthenticationEip4361Nonce * Web3AuthenticationNonceEntity -> Web3AuthenticationEip4361NonceEntity * Web3AuthenticationEip4361NonceEntity -> Web3AuthEip4361NonceEntity * Web3AuthNonceRepository -> Web3AuthEip4361NonceRepository * Web3NonceService -> Web3AuthEip4361NonceService * Web3NonceServiceImpl -> Web3AuthEip4361NonceServiceImpl * Add auth_eip4361 to filenames * EvmWalletAddressConvertor: tests * Web3AuthEip4361NonceRepository::consume_nonce(): implement * Web3WalletAuthenticationProvider: verify nonce * PostgresWeb3AuthNonceRepository -> PostgresWeb3AuthEip4361NonceRepository * InMemoryWeb3AuthNonceRepository -> InMemoryWeb3AuthEip4361NonceRepository * SqliteWeb3AuthNonceRepository -> SqliteWeb3AuthEip4361NonceRepository * kamu-cli: register db repos * kamu-cli: move kamu_adapter_auth_web3::register_dependencies() to configure_server_catalog() * Changes after merging * test_login_enabled_methods(): fix * Self-review * Fix typos * clippy fixes * GQL: Account::account_type(): add * Account::prepare_account_name_for_storage(): save checksummed wallet address caseness * odf::AccountID: the type as an enum (initial migration) * AccountID: as_did() -> as_did_odf() * DidPkh: implement * odf::AccountID: tests * Web3WalletAuthenticationProvider::login(): generate did:pkh: account ID * Fixes after merging * test_read_shapefile_geom(): fix test * Web3WalletAuthenticationProvider: tests * Self-review * odf::AccountID::to_stack_string(): implement * odf::AccountID::as_stack_string(): implement * ToStackString: implement * odf::AccountID::as_id_without_did_prefix(): implement * Self-review * Self-review [2] * AccountProvider: introduce * lazy_static: remove dep * GQL: Account::account_provider(): return AccountProvider enum * GQL: Eip4361AuthNonce: scalar * Web3WalletAuthenticationProviderHarness: add signature generation notes * test_signature_verified(): add * kamu-cli: register kamu_adapter_auth_web3 with kamu_adapter_oauth * AccountService: create_account() -> create_password_account() * LoginPasswordAuthProvider::login(): update comment * Web3AuthenticationEip4361Nonce: use regex * opendatafabric-metadata: add "did-pkh" feature * kamu-accounts-repo-tests: remove unused dep * kamu-cli: fix "web-ui" build * GQL: AuthMut::login(): use AccountProvider for "login_method" argument * Unittests fixes * CHANGELOG.md: update * Release (minor): 0.240.0 (#1255) * Update `sqlx` to `0.8.6`, vol.2 (#1222) * sqlx: 0.8.5 * images: [email protected] * CHANGELOG.md: update * sqlx: 0.8.6 * sqlx: 0.8.6 [2] * images/sqlx-cli: read versions from the repo * Hotfix: `web3-wallet` authorization provider: interactive login use case support (Device Flow) (#1257) * Web3WalletAuthenticationProvider: fix device flow * Release (patch): 0.240.1 * Post-merge changes --------- Co-authored-by: Sergii Mikhtoniuk <[email protected]> Co-authored-by: Roman Boiko <[email protected]>
When creating datasets on user's behalf, ensure naming scheme for the users matches parent account name.
* Revert migration versions renaming * Fix yanked crate: curve25519-dalek --------- Co-authored-by: Dima Pristupa <[email protected]>
…#1319) * kamu-adapter-graphql: add U256 scalar * MoleculeMut::create_project(): use U256 type for ipnft_token_id * schema.gql: update * MoleculeProject::ipnft_token_id: use BigInt for backward compatibility * schema.gql: update
* observability: make dill dependency optional (#1300) * Support flow retries (#1292) * Sketched core changes in flow system to support retries * Modeling retry policy as an optional part of `FlowConfiguration` (independent on flow type) * Simplifications of GQL API for flow configurations + first attempt to add retries view * Using Option<RetryPolicy> instead of default object * GQL API sketched to set retry policy with config * `setConfig` method in API replaced with type-safe `setIngestConfig` and `setCompactionConfig` with less error-prone situations * `triggerFlow` operation expanded on 4 operations per particular flow types * Published "scheduled_at" property for flow timing * Fixed bug: retry policy not reloading correctly * Bug: don't send flow progress message unless outcome is attached * Bug: reindexing fails if owner is inaccessible * `finished_at` => `last_attempt_finished_at` in flow projection. * Fixed a couple detected bugs in flow agent on retries scheduling and finished task hjandling * Flow API simplified: dataset_id at top level, removed from description objects * Added 'nextAttemptAt' field to `FlowEventTaskChanged` API * GQL: TaskOutcome transports failure reasons, converted FlowFailureReason into TaskFailureReason, and reused for both task and flow outcomes * Optimization for flow listing: `FlowDescriptionDatasetPollingIngest` now provides it's own dedicated copy of relevant `pollingSource` event * Similar optimization for transform result * Tests: retry policy in flow configs * Tests: flow event store with retry attempts * Tests: FlowAgent scenarios with retries * Tests: more sophisticated queries for `match_datasets_having_blocks` in key blocks repository * Tests: GQL: retry triggers, scheduledAt field * Tests: GQL: FlowTimingRecords - initiatedAt * Tests: GQL: optimization for polling source and transform for flow history * Tests: GQL: task-level error reasons * v0.245.0 * Backport collection and files API improvements * Hotfix: performance - avoid loading tasks in flow listings (#1302) * v0.245.1 * Observability crate improvements * v0.245.2 * Open gql playground in anonymous mode (#1304) * Open gql playground in anonymous mode * Update changelog * Add comments * GQL Search also filters by id (#1306) * Use nix to install correct flatc version for codegen lint * Move verbose parameters from spans to log events * Unify root span prefixes * Release v0.245.3 (#1315) * Upgrade to datafusion 49 * Bump the all group with 2 updates Bumps the all group with 2 updates: [cachix/install-nix-action](https://github.com/cachix/install-nix-action) and [cachix/cachix-action](https://github.com/cachix/cachix-action). Updates `cachix/install-nix-action` from 25 to 31 - [Release notes](https://github.com/cachix/install-nix-action/releases) - [Changelog](https://github.com/cachix/install-nix-action/blob/master/RELEASE.md) - [Commits](cachix/install-nix-action@v25...v31) Updates `cachix/cachix-action` from 15 to 16 - [Release notes](https://github.com/cachix/cachix-action/releases) - [Commits](cachix/cachix-action@v15...v16) --- updated-dependencies: - dependency-name: cachix/install-nix-action dependency-version: '31' dependency-type: direct:production update-type: version-update:semver-major dependency-group: all - dependency-name: cachix/cachix-action dependency-version: '16' dependency-type: direct:production update-type: version-update:semver-major dependency-group: all ... Signed-off-by: dependabot[bot] <[email protected]> * Bump actions/download-artifact from 4 to 5 in the all group Bumps the all group with 1 update: [actions/download-artifact](https://github.com/actions/download-artifact). Updates `actions/download-artifact` from 4 to 5 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v4...v5) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major dependency-group: all ... Signed-off-by: dependabot[bot] <[email protected]> * Add extended events block gql api (#1316) * Add extended events block gql api * Update schema * Update changelog * Replace blocks method by a single one * Add push source visitor * Update schema * Update changelog * Remove visitors mapping * Update schema * Move encoding to method level * Correctly handle empty authentication token error (#1324) * Correctly handle empty authentication token error * Change error to 400 * Bump actions/checkout from 4 to 5 in the all group Bumps the all group with 1 update: [actions/checkout](https://github.com/actions/checkout). Updates `actions/checkout` from 4 to 5 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v5) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major dependency-group: all ... Signed-off-by: dependabot[bot] <[email protected]> * Upgrade slab per RUSTSEC-2025-0047 * Patch release `0.245.4` (#1327) * Patch release 0.245.4 * Fix tests --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Sergii Mikhtoniuk <[email protected]> Co-authored-by: Sergei Zaychenko <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
rmn-boiko
force-pushed
the
sku/molecule
branch
from
e5e24fd to
82473cf
Compare
* StackString::make_ascii_lowercase(): implement * GQL: Search::query() * GQL: Search::query(): tests * CHANGELOG.md: update * Update tracing-subscriber * MoleculeMut::project_account_name(): generate lowercase project account name * Add migrations
# Conflicts: # CHANGELOG.md
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
2nd delivery includes:
moleculeGQL API group