sig providers meeting notes #34

sig-providers/meetings/034-2025-10-22.md

@@ -0,0 +1,198 @@
+# Akash Network - Providers Special Interest Group (SIG) - Meeting #34
+
+## Agenda  
+- Discussion on the upcoming Akash Mainnet 14 network upgrade  
+- Updates on Provider Console and JWT authentication implementation  
+- Testing status and release preparations for the Cosmos SDK 53 migration  
+- Provider best practices during the network upgrade  
+- Community questions, clarifications, and resource sharing  
+
+## Meeting Details
+- Date: Wednesday, October 22, 2025
+- Time: 08:00 AM PT (Pacific Time)
+- Recording: Coming Soon
+- [Transcript](#transcript)
+
+
+## Participants  
+- Tyler Wright  
+- Jigar Patel  
+- Deval Patel  
+- Zeke Ezagui  
+- Rene Claude  
+- Ariel Leza  
+- Rodri R  
+- Vignesh V  
+- Alex Pedersen  
+- Damir Simpovic  
+- B S  
+- Deathless  
+- Scott Carruthers  
+
+## Meeting Notes
+
+### Discussion on the upcoming Akash Mainnet 14 network upgrade
+- Target window set for Tuesday, October 28, 2025, around 13:00 UTC; exact time depends on block height and final validations.  
+- Order of operations: validators upgrade first; provider guidance follows only after network stability is confirmed.  
+- Status and timing updates to be announced across Discord and social channels.  
+- Steering Committee meets after the upgrade later that week for a post-event sync.  
+- No overview video; upgrade docs and the on-chain proposal explain rationale, scope, and steps.  
+
+### Updates on Provider Console and JWT authentication implementation
+- Provider Console prioritized for JWT compatibility and the upcoming network changes.  
+- Extensive sandbox testing reported; console flow believed to be ready for the upgrade.  
+- JWT signing already shipped to many providers using the console.  
+- After the network event, focus returns to notification features and other console backlog items.  
+- Provider Services v0.8.4 released with JWT authentication; operators urged to be on this version before the upgrade.  
+- JWT vs mTLS recap: mTLS needs chain availability for cert verification; JWT works off-chain, keeping logs/events/status accessible during chain downtime.  
+- Ariel Leza requested implementation clarity; Zeke to share a reference link for deeper details.
+
+### Testing status and release preparations for the Cosmos SDK 53 migration
+- Migration path from Cosmos SDK 47 to 53 noted as state-breaking but beneficial for performance and reliability.  
+- Team has been testing upgrade paths in sandbox environments with positive results.  
+- Confidence expressed that console and provider flows are aligned with the SDK 53 changes.
+
+### Provider best practices during the network upgrade
+- Do not restart, upgrade, or modify provider pods during the upgrade window; doing so can block pod restart and lock out client access.  
+- Remain on the latest provider release (v0.8.4) ahead of the window to enable offline-mode observability via JWT.  
+- Wait for explicit post-upgrade announcements before taking any provider-side actions.  
+- Test JWT functionality beforehand to reduce support load after the event.
+
+### Community questions, clarifications, and resource sharing
+- Rene Claude asked for an overview resource; guidance given to use the docs and on-chain proposal.  
+- Tyler to share links to upgrade docs and the proposal in relevant Discord channels.  
+- Appreciation expressed to Insiders, Vanguards, and core contributors supporting users and testing releases.  
+- Prior month notes are in PR and will be visible with an added video.
+
+---
+
+## Action Items
+
+### Mainnet 14 network upgrade
+- Tyler: share upgrade documentation and on-chain proposal links; coordinate pre/post-event announcements.  
+- All: monitor Discord and social channels for exact timing and go/no-go updates.
+
+### Provider Console and JWT
+- All providers: upgrade to Provider Services v0.8.4 ahead of the window.  
+- Jigar & Deval: continue console validation; resume notification features after the upgrade.  
+- Zeke: share JWT implementation references and reminders.
+
+### SDK 53 migration readiness
+- All providers: review migration notes in shared docs; validate local procedures align with SDK 53 expectations.  
+- Support contributors: prepare FAQs referencing JWT benefits during chain downtime.
+
+### Best practices during upgrade
+- All providers: avoid touching provider pods during the window; wait for explicit post-upgrade instructions.  
+- Support team: reinforce “no changes” guidance and monitor for incidents related to premature restarts.
+
+### Community resources and follow-ups
+- Community members: review shared docs; raise issues with clear repro steps in the provider console support channel.  
+- Tyler and moderators: keep relevant channels updated and summarize outcomes in the post-upgrade sync.
+
+
+## **Transcript**
+
+Tyler Wright: All Welcome everybody to SIG providers. It is October 22nd, 2025. During the special interest group for providers, the Akash network community comes together to talk about all things related to the AAOSH provider. we oftentimes get updates from Jigger and Dval who were previously working on Praor now the provider console as well as other provider related updates that the core engineering team or others are working on.
+
+Tyler Wright: last month we talked we got an update about all the things that were going on related to provider console and as they're getting ready for JWT authentication and Cosmos SDK 53 migration and doing a lot of testing and then there was some adding of a notification system on the pro provider release side again there was a couple provider releases over the last month the notes for September are currently in PR mode. I just have to add a video, but they are accessible to again anybody in the community. if anybody has anything provider related that they want to make sure gets discussed as a part of the agenda, feel free to drop in something in the chat and we can make sure it gets added.
+
+Tyler Wright: but as we always do, I will hand it over to Jigger and Dval to see if there's anything going on specifically over the last month that they want to talk about. Obviously, we've talked about this in SIG support, and elsewhere. There's the biggest item is the network upgrade to AOS mainet 14.  Again, there's a couple of active environments where a number of folks in the community and the core team and clients are actively building right now and doing some testing and testing again functionality leading up to network upgrades. So, I know that there are some small changes that may be coming soon before the binaries get released this weekend.  The goal and the vote is on chain right now is the network upgrade is planned and scheduled for Tuesday, October 28th at around 1300 UTC.
+
+Tyler Wright: Again, that time is subject to change based upon the block heights and as we get closer to the network upgrade, there will be announcements made on a more approximate time for the upgrade. But again, that's been the big effort. The idea is that validators will do the network upgrade first and then there will be announcements made for when providers can upgrade.  With that said, again, I'll kick it over to Jigger and Deval to see if there's anything that they've been working on or want to discuss as it pertains to the network upgrade or just the provider console in general.
+
+Jigar Patel: Thanks yes so for the past month our main focus was to make sure that our provider console is compatible with JWT upgrade and upcoming network upgrade 14. and then we are working to make sure that there is no issues in the upgrade process. we are testing rigorously right now in our sandbox to make sure that all the provider that use the provider console through provider console and yeah so lately the tests are very successful and we are very confident that the process will go as planned.
+
+Jigar Patel: and for the past month we also pushed JWD upgrade. that also went to our and a lot of provider console users have upgraded. They provided through provide this a big relief and it's working now the JW signing is working on those providers as well.
+
+Jigar Patel: and once that's the big biggest ticket item is the network upgrade 14 and once that is we'll be finished up probably next week Tuesday and we'll be back on track with the notifications at home stuff. That's my update. Thanks D.
+
+
+### 00:05:00
+
+Tyler Wright: Thank you. Does anyone have any questions as pertains to provider console? Again, some of the work that's been done around JWT on the upgrade there as well as anything as we get ready for the Akash mainet 14 network upgrade happening next Tuesday, October 28th.  All right. Again, if anybody has any items there is in Discord, please feel free. I know that Jigger and Dval are oftentimes in Discord along with other members of the Akash Insiders Vanguards and core team. So if anybody has any questions or any items as they're using provider console, please feel free.
+
+Tyler Wright: There's a provider console support channel where people can leave their feedback get involved, ask questions as they're continuing to use a All right. moving along.
+
+Rene Claude: Yes,…
+
+Tyler Wright: Go ahead.
+
+Rene Claude: Rene. How are you?
+
+Tyler Wright: How are you?
+
+Rene Claude: I'm good. I just want to see is there a sort of ramp up article that kind of has an overview on all the details involved with the upgrade like a resource video or otherwise.
+
+Tyler Wright: There's no resource video. I will drop in documentation as well as the proposal that is on chain for vote. Both of those resources have a great deal of information on why the network upgrade is happening. some of the technical components there but there is no resource video.  I think the main takeaway Akash has been on Kas SDK 47 for some time. and again this is an active decision. as Kasos SDK is continuing to grow, there are some things about some SDK versions that were less than desirable.
+
+Tyler Wright: And now again the migration into Kasos SDK 53 with a number of other items that have been added that are again state breaking changes that have been added as a part of the network upgrade. I'll also drop the documentation in the chat. Again the binaries are not there yet but will be closer to network upgrade time. this has a instruction guide for validators and others on the network upgrade. unfortunately there's no other resources available at this time as it pertains to network upgrade.
+
+Tyler Wright: But do you have a specific question or thought that you would hope would be available to you via some sort of resource?
+
+Rene Claude: Yeah. Yeah.
+
+Rene Claude: No, I just wanted the overview. So, I think just reading through those docs should get me up to speed. Thank you so much.
+
+Tyler Wright: Of course. No problem at all. Cool.
+
+Tyler Wright: So again, if anybody has any questions about the network upgrade, feel free to use the SIG chain channel. if it's provider related, again, feel free to use the provider channel and folks from the inside our Vanguard community can make sure that the appropriate folks are answering those kinds of questions. All right. next up I'll kick it over to Zeke Scott to see if there's any provider related items that they want to make sure get discussed prior to the network upgrade or anything on the provider side that they're seeing that might be worth calling out.
+
+Zeke Ezagui: Sure, so yeah, a couple weeks ago, I believe it was about two and a half weeks ago, we released provider services 0.8.4, which is the JWT release. so if you haven't please upgrade as this will enable users of AOS to switch to JWT authentication instead of NTLS which will allow them to access lease logs lease events and deployment status via straight from the provider pod.  So, if the chain is offline like it will be for this upgrade, those clients will still be able to use those features. if you have any questions, drop them in Discord and you can ask.  Now back to you,
+
+
+### 00:10:00
+
+Tyler Wright: Perfect. Becky, Zeke, again, just that's an important call out during the network upgrade, this goes for every network upgrade. that have active deployments on the network, those will be unfased. the challenge will be around again deploying new items during that time. there will be notifications made across the website social media channels to let people know when the network upgrade is happening and then when it is over.
+
+Tyler Wright: But again that JWT authentication will allow for a number of key clients and everybody in the community to kind of track things that are going on in terms of inside the provider during the time the network upgrade is happening. So if you have not upgraded to the most recent version of the provider release, please do so over the next couple of days if not sooner. Anyone have any questions about anything provider release?
+
+Ariel Leza: Yeah, I want to add something just to better understand.
+
+Tyler Wright: right.
+
+Ariel Leza: First of all, hi, nice to meet you guys. My name is Ariel. It's the first time that I've finally joined the provider call. so I just want to ask because I hear you guys repeating about the GWT feature. the reason there will be access to additional logs is because you're changing the authentication to JWT and essentially that provides accessing a similar identity to using the private key of the provider just to better understand implementation change. Mhm.
+
+Zeke Ezagui: So basically MTLS uses the blockchain to verify the C. So if the blockchain's down, you can't verify the C.
+
+Zeke Ezagui: JWT doesn't use that. I can drop the AE U also…
+
+Ariel Leza: Okay.
+
+Zeke Ezagui: which has more implementation specifics for
+
+Ariel Leza: Yeah, that'd be great. Thank you. Thanks,
+
+Tyler Wright: Thank you, while Zeke's putting that together, does anyone else have any questions about anything that was discussed today? Again, we always take these opportunities. Sometimes these meeting Oftent times these meetings actually sometimes run over. So, please feel free to ask anything about the provider experience. I hear a hand up or no, maybe Zeke just dropped in a link in the chat. Go. does anyone have any questions as it pertains to again provider console, provider releases, the provider experience in general? All right.
+
+Tyler Wright: Yeah, I'm just trying to look in some of the provider channels inside Discord to see if there's any interesting topics that anybody wants, but it looks like just people are asking questions. and again, folks in the community and folks in the core team are just providing them with answers. So, big shout out to I see a number of folks here on this call. I won't shout you out by name, but a number of you do a great job of helping out community members on with provider experience, testing out provider releases. So, I do want to give you all a round of applause and…
+
+Ariel Leza: pressure.
+
+Tyler Wright: a heart for all the work that you all do. So, thank you for the continued effort and support. Feel like I keep hearing a hand raised, but then I don't see anyone with raise your hand.  Does anyone else have anything that they want to Cool. If not, then again, I just want to make sure they'll be connected prior to the network upgrade. the steering committee, that's usually the fourth Thursday. October is a longer month. So again, the steering committee will be next Thursday after the network upgrade.
+
+Tyler Wright: just as a reminder from an order operations standpoint as I said at the top of the call again validators will upgrade first then members of the community validators and the core team will make a determination when it's determined that the network is stable and then announcements will be made for providers to take necessary steps.  So on Tuesday, just look out for a number of announcements whether it be from the core team the folks behind provider console etc. when the providers need to do all they need to do post network upgrade. But please wait for announcements before taking any next steps and should be good to go from there.
+
+
+### 00:15:00
+
+Tyler Wright: really looking forward to a successful network upgrade and then again with Cosmos SDK 53 a lot of things will be unlocked in a way that should speed up momentum. Go ahead Zeke.
+
+Zeke Ezagui: So, we'll make further announcements about this, but like Tyler was saying, it's very important. when the upgrade's happening, you don't touch your provider. And the reason being is because if the provider pod is messed with, it won't be able to start back up, which means no one will be like clients won't be able to access their leases on your provider. So, if you don't touch your provider, it'll stay running, which will allow clients to use offline mode and all that. But if you try to upgrade early, you try to, bounce the provider pod do anything like that, you'll basically lock out users from using offline mode. So please don't mess with your provider unless you want angry clients. It's basically what that is. So appreciate it.
+
+Zeke Ezagui: Have a great rest of your day, everybody.
+
+Tyler Wright: Perfect. Thank you,…
+
+Tyler Wright: Again, if anybody has any questions, please drop them in. You can use SIG providers channel. You can use any of the channels inside the provider section, Discord. and then again, feel free to tag myself and insider, Vanguard, or another member of the core team, and we'll get you prompt answers as we get ready for Tuesday, October 28th. Again, appreciate everyone's time.
+
+Tyler Wright: M Jigger, as always, Zek thank you for the updates on some of the provider releases and core. and again, hope everyone has a great rest of their day. and excited to talk to everybody post upgrade. Again, as Zeke has said, leading up to network upgrade, make sure that you're on the most recent provider release if you aren't already. And then during network upgrade, just hold tight and just wait for instruction. That's all we need.  But much appreciate this group. again, pass on any messaging that you see for folks that couldn't make it and thanks for all those that are listening later. Have a great rest of your day. and again, looking forward to talking to you all post network upgrade next week. Thank you very much everybody.
+
+Deval Patel: Okay, guys.
+
+Rodri R: Thank you, Tyler and Vice.
+
+Vignesh V: Thank you.
+
+
+### Meeting ended after 00:20:37 👋
+