Large modpacks trigger a SIGABRT

[mmtawous]

Currently, large modpacks such as the Versatile v7.5 pack triggers a SIGABRT caused by the following

runtime/cgo: pthread_create failed: Resource temporarily unavailable

This is caused by the value in the --dl-threads flag not being used in the Downloader. Here is a snippet of the relevant code from main.

mrpack-install/web/download/multi.go

Lines 20 to 38 in 91991df

	type Downloader struct {
	Downloads []*Download
	Threads int // TODO
	Retries int
	}
	func (g *Downloader) Download(baseDir string) {
	var wg sync.WaitGroup
	for i := range g.Downloads {
	wg.Add(1)
	dl := g.Downloads[i]
	go func() {
	defer wg.Done()
	absPath, _ := filepath.Abs(filepath.Join(baseDir, dl.Path))
	success := false
	for _, link := range dl.Urls {
	// retry when download failed
	for retries := 0; retries < g.Retries; retries++ {
	// try download

[nothub]

Resolved 🚀

[nothub]

On a sidenote, I tried to reproduce the issue once on my system but I was not able to reproduce it.
Can you post the DefaultTasksMax (systemctl show --property=DefaultTasksMax) on the system you experienced the issue on?

[mmtawous]

I am running on an Apple Silicon M2 Macbook Air running MacOS 13.5.1. MacOS doesn't have systemctl so here is what I think is the closest thing to what you are referring to. Oddly enough I don't think that modpack had 4096 things to download so maybe this isn't what I am looking for.

~  ulimit -u
2666

~  sysctl kern.num_threads
kern.num_threads: 20480

~  sysctl kern.num_taskthreads
kern.num_taskthreads: 4096

Adding print statements to the code before the fix doesn't give any more useful info. All 261 routines (there are 261 dependencies) appear to be started before it crashes. I am unfamiliar with how Go handles threads, so I can't offer any more insight into what's happening.

[nothub]

Thanks for the infos, I gonna test around in a ressource constrained VM.