Improve resource allocation choice generator #6313

yuvipanda · 2025-07-02T20:36:26Z

In GHG: tune the CPU request for SSIM workshop #6309, we discovered that some of our resource allocations were now outdated, causing some servers to not spawn. We add a 2% wiggle room (clamped at 2GB / 1CPU) to work around this.
Based on feedback from earthscope's @sarahwilson523 in earthscope: Rename the resource request labels #6351 (comment), we've updated the choice descriptions to be clearer to end users. No more fractions, better messaging around CPU guarantees and limits.
Allow generating allocations for multiple instance types in one go, to reduce copy pasting

In 2i2c-org#6309, we discovered that some of our resource allocations were now outdated. This adds a 2% wiggle room that should help with that

Otherwise we were losing a lot of resources on large nodes

Based on 2i2c-org#6351 (comment) 1. Much more limited use of fractions - only for CPUs under 2. No fractions over that, everything is rounded to nearest whole number 2. The display name indicates how much CPU is guaranteed 3. The description indicates CPU limits with appropriate wording 4. Allow specifying multiple node types in one go, makes copy pasting easier. 5. No longer specify 'default: true', as first one is default now

for more information, see https://pre-commit.ci

Based on conversation in 2i2c-org#6351 Regenerated based on 2i2c-org#6313

github-actions · 2025-07-15T20:08:52Z

Merging this PR will trigger the following deployment actions.

Support deployments

No support upgrades will be triggered

Staging deployments

Cloud Provider	Cluster Name	Hub Name	Reason for Redeploy
aws	victor	staging	Core infrastructure has been modified
aws	neurohackademy	staging	Core infrastructure has been modified
aws	nasa-cryo	staging	Core infrastructure has been modified
aws	reflective	staging	Core infrastructure has been modified
aws	nasa-veda	staging	Core infrastructure has been modified
gcp	2i2c	staging	Core infrastructure has been modified
gcp	2i2c	dask-staging	Core infrastructure has been modified
gcp	2i2c	ucmerced-staging	Core infrastructure has been modified
aws	maap	staging	Core infrastructure has been modified
aws	disasters	staging	Core infrastructure has been modified
gcp	cloudbank	staging	Core infrastructure has been modified
gcp	climatematch	staging	Core infrastructure has been modified
aws	2i2c-aws-us	staging	Core infrastructure has been modified
aws	2i2c-aws-us	dask-staging	Core infrastructure has been modified
aws	earthscope	staging	Core infrastructure has been modified
aws	openscapes	staging	Core infrastructure has been modified
gcp	hhmi	staging	Core infrastructure has been modified
kubeconfig	2i2c-jetstream2	staging	Core infrastructure has been modified
aws	strudel	staging	Core infrastructure has been modified
aws	ubc-eoas	staging	Core infrastructure has been modified
aws	nmfs-openscapes	staging	Core infrastructure has been modified
gcp	catalystproject-latam	staging	Core infrastructure has been modified
aws	opensci	staging	Core infrastructure has been modified
aws	nasa-ghg	staging	Core infrastructure has been modified
aws	jupyter-health	staging	Core infrastructure has been modified
gcp	awi-ciroh	staging	Core infrastructure has been modified
aws	catalystproject-africa	staging	Core infrastructure has been modified
kubeconfig	utoronto	staging	Core infrastructure has been modified
kubeconfig	utoronto	r-staging	Core infrastructure has been modified
aws	projectpythia	staging	Core infrastructure has been modified
gcp	2i2c-uk	staging	Core infrastructure has been modified
aws	smithsonian	staging	Core infrastructure has been modified
gcp	leap	staging	Core infrastructure has been modified

Production deployments

Cloud Provider	Cluster Name	Hub Name	Reason for Redeploy
gcp	dubois	ephemeral	Core infrastructure has been modified
aws	victor	prod	Core infrastructure has been modified
aws	neurohackademy	prod	Core infrastructure has been modified
aws	nasa-cryo	prod	Core infrastructure has been modified
aws	reflective	prod	Core infrastructure has been modified
aws	reflective	workshop	Core infrastructure has been modified
aws	nasa-veda	prod	Core infrastructure has been modified
aws	nasa-veda	binder	Core infrastructure has been modified
gcp	2i2c	imagebuilding-demo	Core infrastructure has been modified
gcp	2i2c	binderhub-ui-demo	Core infrastructure has been modified
gcp	2i2c	demo	Core infrastructure has been modified
gcp	2i2c	temple	Core infrastructure has been modified
gcp	2i2c	ucmerced	Core infrastructure has been modified
gcp	2i2c	mtu	Core infrastructure has been modified
aws	maap	prod	Core infrastructure has been modified
aws	disasters	prod	Core infrastructure has been modified
gcp	cloudbank	authoring	Core infrastructure has been modified
gcp	cloudbank	bcc	Core infrastructure has been modified
gcp	cloudbank	chaffey	Core infrastructure has been modified
gcp	cloudbank	ccc	Core infrastructure has been modified
gcp	cloudbank	ccsf	Core infrastructure has been modified
gcp	cloudbank	chabot	Core infrastructure has been modified
gcp	cloudbank	csm	Core infrastructure has been modified
gcp	cloudbank	csueb	Core infrastructure has been modified
gcp	cloudbank	csuf	Core infrastructure has been modified
gcp	cloudbank	csula	Core infrastructure has been modified
gcp	cloudbank	csulb	Core infrastructure has been modified
gcp	cloudbank	csun	Core infrastructure has been modified
gcp	cloudbank	csum	Core infrastructure has been modified
gcp	cloudbank	csumb	Core infrastructure has been modified
gcp	cloudbank	csus	Core infrastructure has been modified
gcp	cloudbank	demo	Core infrastructure has been modified
gcp	cloudbank	dvc	Core infrastructure has been modified
gcp	cloudbank	elac	Core infrastructure has been modified
gcp	cloudbank	elcamino	Core infrastructure has been modified
gcp	cloudbank	evc	Core infrastructure has been modified
gcp	cloudbank	fresno	Core infrastructure has been modified
gcp	cloudbank	foothill	Core infrastructure has been modified
gcp	cloudbank	glendale	Core infrastructure has been modified
gcp	cloudbank	golden	Core infrastructure has been modified
gcp	cloudbank	high	Core infrastructure has been modified
gcp	cloudbank	howard	Core infrastructure has been modified
gcp	cloudbank	humboldt	Core infrastructure has been modified
gcp	cloudbank	lacc	Core infrastructure has been modified
gcp	cloudbank	lahc	Core infrastructure has been modified
gcp	cloudbank	lamission	Core infrastructure has been modified
gcp	cloudbank	laney	Core infrastructure has been modified
gcp	cloudbank	lavc	Core infrastructure has been modified
gcp	cloudbank	lbcc	Core infrastructure has been modified
gcp	cloudbank	mendocino	Core infrastructure has been modified
gcp	cloudbank	merced	Core infrastructure has been modified
gcp	cloudbank	merritt	Core infrastructure has been modified
gcp	cloudbank	mills	Core infrastructure has been modified
gcp	cloudbank	miracosta	Core infrastructure has been modified
gcp	cloudbank	mission	Core infrastructure has been modified
gcp	cloudbank	moreno	Core infrastructure has been modified
gcp	cloudbank	norco	Core infrastructure has been modified
gcp	cloudbank	palomar	Core infrastructure has been modified
gcp	cloudbank	pasadena	Core infrastructure has been modified
gcp	cloudbank	redwoods	Core infrastructure has been modified
gcp	cloudbank	reedley	Core infrastructure has been modified
gcp	cloudbank	riohondo	Core infrastructure has been modified
gcp	cloudbank	sacramento	Core infrastructure has been modified
gcp	cloudbank	saddleback	Core infrastructure has been modified
gcp	cloudbank	santiago	Core infrastructure has been modified
gcp	cloudbank	sbcc	Core infrastructure has been modified
gcp	cloudbank	sbcc-dev	Core infrastructure has been modified
gcp	cloudbank	sierra	Core infrastructure has been modified
gcp	cloudbank	sjcc	Core infrastructure has been modified
gcp	cloudbank	sjsu	Core infrastructure has been modified
gcp	cloudbank	skyline	Core infrastructure has been modified
gcp	cloudbank	srjc	Core infrastructure has been modified
gcp	cloudbank	tuskegee	Core infrastructure has been modified
gcp	cloudbank	ucsc	Core infrastructure has been modified
gcp	cloudbank	wlac	Core infrastructure has been modified
gcp	climatematch	prod	Core infrastructure has been modified
aws	2i2c-aws-us	showcase	Core infrastructure has been modified
aws	earthscope	prod	Core infrastructure has been modified
aws	earthscope	binder	Core infrastructure has been modified
aws	openscapes	prod	Core infrastructure has been modified
aws	openscapes	workshop	Core infrastructure has been modified
gcp	hhmi	spyglass	Core infrastructure has been modified
gcp	hhmi	binder	Core infrastructure has been modified
kubeconfig	projectpythia-binder	binderhub	Core infrastructure has been modified
aws	strudel	prod	Core infrastructure has been modified
aws	ubc-eoas	prod	Core infrastructure has been modified
aws	nmfs-openscapes	prod	Core infrastructure has been modified
aws	nmfs-openscapes	workshop	Core infrastructure has been modified
aws	nmfs-openscapes	noaa-only	Core infrastructure has been modified
gcp	catalystproject-latam	unitefa-conicet	Core infrastructure has been modified
gcp	catalystproject-latam	cicada	Core infrastructure has been modified
gcp	catalystproject-latam	gita	Core infrastructure has been modified
gcp	catalystproject-latam	iner	Core infrastructure has been modified
gcp	catalystproject-latam	plnc	Core infrastructure has been modified
gcp	catalystproject-latam	unam	Core infrastructure has been modified
gcp	catalystproject-latam	cabana	Core infrastructure has been modified
gcp	catalystproject-latam	nnb-ccg	Core infrastructure has been modified
gcp	catalystproject-latam	labi	Core infrastructure has been modified
gcp	catalystproject-latam	areciboc3	Core infrastructure has been modified
gcp	catalystproject-latam	valledellili	Core infrastructure has been modified
aws	opensci	sciencecore	Core infrastructure has been modified
aws	opensci	climaterisk	Core infrastructure has been modified
aws	opensci	small-binder	Core infrastructure has been modified
aws	opensci	big-binder	Core infrastructure has been modified
aws	nasa-ghg	prod	Core infrastructure has been modified
aws	nasa-ghg	binder	Core infrastructure has been modified
aws	jupyter-health	prod	Core infrastructure has been modified
gcp	awi-ciroh	prod	Core infrastructure has been modified
gcp	awi-ciroh	workshop	Core infrastructure has been modified
aws	catalystproject-africa	nm-aist	Core infrastructure has been modified
aws	catalystproject-africa	must	Core infrastructure has been modified
aws	catalystproject-africa	uvri	Core infrastructure has been modified
aws	catalystproject-africa	wits	Core infrastructure has been modified
aws	catalystproject-africa	kush	Core infrastructure has been modified
aws	catalystproject-africa	molerhealth	Core infrastructure has been modified
aws	catalystproject-africa	aibst	Core infrastructure has been modified
aws	catalystproject-africa	bhki	Core infrastructure has been modified
aws	catalystproject-africa	bon	Core infrastructure has been modified
kubeconfig	utoronto	prod	Core infrastructure has been modified
kubeconfig	utoronto	r-prod	Core infrastructure has been modified
kubeconfig	utoronto	highmem	Core infrastructure has been modified
aws	projectpythia	prod	Core infrastructure has been modified
aws	projectpythia	pythia-binder	Core infrastructure has been modified
gcp	2i2c-uk	lis	Core infrastructure has been modified
aws	smithsonian	prod	Core infrastructure has been modified
gcp	leap	prod	Core infrastructure has been modified
gcp	leap	public	Core infrastructure has been modified

GeorgianaElena · 2025-07-30T10:17:57Z

@yuvipanda, I know that this could be improved/documented as part of #6213, BUT since it looks we haven't had time to commit it these past sprints, can we merge this PR as it is so we can use it more easily?

* docs: improve deployment information * feat: add new hub * Update config/clusters/berkeley-geojupyter/cluster.yaml * feat: use resource allocation from #6313

yuvipanda · 2025-08-06T20:48:43Z

Fair enough, @GeorgianaElena - merged.

* docs: improve deployment information * feat: add new hub * Update config/clusters/berkeley-geojupyter/cluster.yaml * feat: use resource allocation from 2i2c-org#6313

yuvipanda mentioned this pull request Jul 2, 2025

GHG: tune the CPU request for SSIM workshop #6309

Merged

yuvipanda added a commit to yuvipanda/pilot-hubs that referenced this pull request Jul 2, 2025

Adjust earthscope limits a little

0db21f8

Brings in 2i2c-org#6313

yuvipanda mentioned this pull request Jul 2, 2025

Adjust earthscope limits a little #6314

Closed

yuvipanda mentioned this pull request Jul 14, 2025

earthscope: Rename the resource request labels #6351

Open

yuvipanda added 3 commits July 15, 2025 10:57

Add 2% wiggle room to resource allocations

8ecb07e

In 2i2c-org#6309, we discovered that some of our resource allocations were now outdated. This adds a 2% wiggle room that should help with that

Clamp wiggle room at 2GB / 2 CPU

a4b10ca

Otherwise we were losing a lot of resources on large nodes

yuvipanda force-pushed the resource-alloc-wiggle branch from daab30e to ceedd54 Compare July 15, 2025 20:06

[pre-commit.ci] auto fixes from pre-commit.com hooks

24a2342

for more information, see https://pre-commit.ci

yuvipanda added a commit to yuvipanda/pilot-hubs that referenced this pull request Jul 15, 2025

Deploy clearer resource allocation labels to earthscope

36e8f4b

Based on conversation in 2i2c-org#6351 Regenerated based on 2i2c-org#6313

yuvipanda mentioned this pull request Jul 15, 2025

[Merge after July 29, 2025] Deploy clearer resource allocation labels to earthscope #6366

Merged

yuvipanda marked this pull request as draft July 15, 2025 20:12

yuvipanda changed the title ~~Add 2% wiggle room to resource allocations~~ Improve resource allocation choice generator Jul 16, 2025

yuvipanda added 3 commits July 16, 2025 11:51

Slightly adjust wording on generated choices

2c2a806

Remove unused import

b49c69d

Name some constants

f8d3ddf

GeorgianaElena mentioned this pull request Jul 17, 2025

[openscapes] Offer just one allocation option for the workshop on a bigger machine: #6381

Merged

yuvipanda mentioned this pull request Jul 25, 2025

[New Hub] [Phase 3] Hub Setup - berkeley-geojupyter #6454

Closed

GeorgianaElena mentioned this pull request Jul 30, 2025

[catalystproject-latam] Profile options don't have node selectors #6480

Open

7 tasks

agoose77 added a commit that referenced this pull request Aug 6, 2025

feat: use resource allocation from #6313

55a741e

agoose77 added a commit that referenced this pull request Aug 6, 2025

[new hub] GeoJupyter staging and prod (#6492)

b287853

* docs: improve deployment information * feat: add new hub * Update config/clusters/berkeley-geojupyter/cluster.yaml * feat: use resource allocation from #6313

yuvipanda marked this pull request as ready for review August 6, 2025 20:48

yuvipanda merged commit effe777 into 2i2c-org:main Aug 6, 2025
42 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve resource allocation choice generator #6313

Improve resource allocation choice generator #6313

Uh oh!

yuvipanda commented Jul 2, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 15, 2025 •

edited

Loading

Uh oh!

GeorgianaElena commented Jul 30, 2025 •

edited

Loading

Uh oh!

Uh oh!

yuvipanda commented Aug 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Improve resource allocation choice generator #6313

Improve resource allocation choice generator #6313

Uh oh!

Conversation

yuvipanda commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Support deployments

Staging deployments

Production deployments

Uh oh!

GeorgianaElena commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

yuvipanda commented Aug 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yuvipanda commented Jul 2, 2025 •

edited

Loading

github-actions bot commented Jul 15, 2025 •

edited

Loading

GeorgianaElena commented Jul 30, 2025 •

edited

Loading