feat: implement token lists management with fetcher and parser support #10

saledjenic · 2025-09-01T16:31:37Z

Added new package tokenlists for managing token lists.
Implemented tokensList struct for thread-safe state management.
Introduced refreshWorker for background updates of token lists.
Created fetchers for remote token lists and individual token lists.
Added parsers for CoinGecko, Uniswap and Status list token formats.
Included configuration options for token lists management.
Implemented validation for token lists against JSON schemas.
Added comprehensive tests for all new functionalities.

codecov-commenter · 2025-09-01T16:35:39Z

Codecov Report

❌ Patch coverage is 76.62722% with 237 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.96%. Comparing base (38d7f35) to head (52bb6d6).

Files with missing lines	Patch %	Lines
pkg/tokenlists/tokenslist.go	59.14%	109 Missing and 25 partials ⚠️
pkg/tokenlists/test_helper.go	60.95%	28 Missing and 13 partials ⚠️
pkg/tokenlists/refresh.go	75.34%	13 Missing and 5 partials ⚠️
pkg/tokenlists/validate.go	79.06%	14 Missing and 4 partials ⚠️
pkg/tokenlists/httpclient.go	78.94%	8 Missing and 4 partials ⚠️
...g/tokenlists/remote_list_of_token_lists_fetcher.go	82.69%	6 Missing and 3 partials ⚠️
pkg/tokenlists/fetcher.go	93.47%	1 Missing and 2 partials ⚠️
pkg/tokenlists/test_data_lists_of_token_lists.go	80.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master      #10       +/-   ##
===========================================
+ Coverage   19.64%   45.96%   +26.32%     
===========================================
  Files          17       34       +17     
  Lines        1181     2195     +1014     
===========================================
+ Hits          232     1009      +777     
- Misses        945     1125      +180     
- Partials        4       61       +57

Files with missing lines	Coverage Δ
pkg/tokenlists/config.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/defaults.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/parser_coingecko_all_tokens.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/parser_standard.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/parser_status.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/remote_token_lists_fetcher.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/test_data_status_token_list.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/test_data_uniswap_token_list.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/types.go	`100.00% <100.00%> (ø)`
pkg/tokenlists/test_data_lists_of_token_lists.go	`80.00% <80.00%> (ø)`
... and 7 more

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

dlipicar

Thanks for splitting the logic into files. I feel we could go one step further and create packages with self-contained single-responsibility functionality, that way these modules become reusable in a more granular level.
I haven't though too much about it, but I imagine something like:

A package in charge of fetching/parsing lists of token lists
A package in charge of fetching/parsing token lists (doesn't care whether they come from, could be downloaded or a local file selected by the user)
A package in charge of merging multiple token lists

The "Service" part that runs the timers and triggers periodical fetches. Hmm I've avoided putting such stuff in the wallet-sdk for now, and I haven't thought too much about it yet, but my general feeling is that something like that belongs more in the consumer (ie status-go) than the wallet-sdk. Anyway, it's a discussion I'd like to have!

dlipicar · 2025-09-11T20:46:39Z

pkg/tokenlists/config.go

+	return c
+}
+
+func (c *Config) WithCommunityTokenStore(store CommunityTokenStore) *Config {


haven't finished reviewing, but this is a red flag. The concept of a community is something status-specific, it most likely doesn't belong in the wallet-sdk

The concept of a community is something status-specific

Yes it is, and the wallet-sdk is here to simplify Status' needs. Was thinking whether to keep it on the stautsgo side or not, but then moved it here and let tokenlists package to take care of everything token-related.

dlipicar · 2025-09-11T21:00:02Z

pkg/tokenlists/config.go

+
+	"go.uber.org/zap"
+)
+


the config struct should probably be defined here for clarity

It can be, but added it to types.go since all types for the package are there.

dlipicar · 2025-09-12T01:19:48Z

pkg/common/chainid.go

@@ -1,5 +1,15 @@
 package common

+const (
+	EthereumNativeCrossChainID = "eth-native"


hmm if this really needs to be in common, it should probably be in a separate file.
My impression is this is token-related, so it should be in some token-related package

Yes, can be moved there.

dlipicar · 2025-09-12T01:23:14Z

pkg/tokenlists/types.go

+	LogoURI      string             `json:"logoUri"`
+
+	CustomToken   bool           `json:"custom"`
+	CommunityData *CommunityData `json:"communityData,omitempty"`


yeah this definitely doesn't belong here. If Status wants community data for some specific token it should get it from some community service.

As said in the previous comment, was thinking which approach to go with, but then wallet-sdk is here mainly for our needs and is aware of custom tokens and community tokens and knows how to handle them if they are provided. If the client doesn't need them, there are default custom and community token storer that can be used. In that case the package behaves as a regular token manager.

but then wallet-sdk is here mainly for our needs

I think we should treat this as a general-purpose library (that’s what an SDK is for). Ideally it should stay flexible enough so it can also be used in the Market-Proxy or even in Logos core, without assumptions about Status-client specific needs.

as we claim in https://github.com/status-im/go-wallet-sdk/blob/master/docs/specs.md

Go Wallet SDK is a modular Go library intended to support the development of multi‑chain cryptocurrency wallets and blockchain applications. The SDK exposes self‑contained packages for common wallet tasks such as fetching account balances across many EVM chains and interacting with Ethereum JSON‑RPC.

- Added new package `tokenlists` for managing token lists. - Implemented `tokensList` struct for thread-safe state management. - Introduced `refreshWorker` for background updates of token lists. - Created fetchers for remote token lists and individual token lists. - Added parsers for CoinGecko, Uniswap and Status list token formats. - Included configuration options for token lists management. - Implemented validation for token lists against JSON schemas. - Added comprehensive tests for all new functionalities.

saledjenic · 2025-09-15T07:22:00Z

I feel we could go one step further and create packages with self-contained single-responsibility functionality, that way these modules become reusable in a more granular level.

We can, ofc, if there is a need for that. That's how it is organized in stautsgo now.

saledjenic requested review from alaibe, friofry and dlipicar September 1, 2025 16:31

saledjenic force-pushed the feat/token-list branch from f3313c0 to b07c664 Compare September 1, 2025 16:34

saledjenic force-pushed the feat/token-list branch 6 times, most recently from 7c0e25d to 15d894a Compare September 8, 2025 09:25

saledjenic force-pushed the feat/token-list branch 2 times, most recently from 8753428 to a572056 Compare September 11, 2025 08:52

dlipicar reviewed Sep 12, 2025

View reviewed changes

saledjenic force-pushed the feat/token-list branch from a572056 to 52bb6d6 Compare September 12, 2025 06:51

saledjenic requested a review from a team as a code owner September 12, 2025 06:51

saledjenic mentioned this pull request Sep 12, 2025

feat!: a new token list approach - wallet sdk token lists integration status-im/status-go#6912

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: implement token lists management with fetcher and parser support #10

feat: implement token lists management with fetcher and parser support #10

Uh oh!

saledjenic commented Sep 1, 2025

Uh oh!

codecov-commenter commented Sep 1, 2025 •

edited

Loading

Uh oh!

dlipicar left a comment

Uh oh!

dlipicar Sep 11, 2025

Uh oh!

saledjenic Sep 15, 2025

Uh oh!

dlipicar Sep 11, 2025

Uh oh!

saledjenic Sep 15, 2025

Uh oh!

dlipicar Sep 12, 2025

Uh oh!

saledjenic Sep 15, 2025

Uh oh!

dlipicar Sep 12, 2025

Uh oh!

saledjenic Sep 15, 2025

Uh oh!

friofry Sep 15, 2025

Uh oh!

friofry Sep 15, 2025

Uh oh!

saledjenic commented Sep 15, 2025

Uh oh!

Uh oh!

feat: implement token lists management with fetcher and parser support #10

Are you sure you want to change the base?

feat: implement token lists management with fetcher and parser support #10

Uh oh!

Conversation

saledjenic commented Sep 1, 2025

Uh oh!

codecov-commenter commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dlipicar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

saledjenic commented Sep 15, 2025

Uh oh!

Uh oh!

codecov-commenter commented Sep 1, 2025 •

edited

Loading